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^  G  A  O 

^^^^^^^Accountabllity  *  tntegrity  *  Reliability _ 

United  States  General  Accounting  Office 
Washington,  DC  20548 


November  27, 2001 

The  Honorable  Fred  Thompson 
Ranking  Minority  Member 
Committee  on  Governmental  Affairs 
United  States  Senate 


Dear  Senator  Thompson: 

As  you  requested,  we  reviewed  the  National  Aeronautics  and  Space 
Administration’s  (NASA)  fiscal  year  2002  performance  plan  required  by  the 
Government  Performance  and  Results  Act  of  1993  (GPRA)‘  to  assess  the 
agency’s  planned  performance  for  the  selected  key  outcomes  that  you 
identified  as  important  mission  areas  for  the  agency/  'These  are  the  same 
outcomes  we  addressed  in  our  June  2000  review  of  NASA’s  fiscal  year  1999 
performance  report  and  fiscal  year  2001  performance  plan  to  provide  a 
baseline  by  which  to  measure  the  agency’s  performance  from  year  to  year/ 
We  also  addressed  these  outcomes  in  our  July  2001  review  of  NASA’s 
fiscal  year  2000  performance  report.^  As  agreed  with  your  office,  we  are 
issuing  this  separate  report  on  our  assessment  of  the  selected  outcomes  in 
NASA’s  fiscal  year  2002  performance  plan.  The  selected  key  outcomes  are 
to 

•  expand  scientific  knowledge  of  the  Earth  system, 

•  expand  the  commercial  development  of  space,  and 

•  deploy  and  operate  the  International  Space  Station  (ISS)  safely  and  cost 
effectively.® 


‘P.L.  103-62, 107Stat.285. 

^This  report  is  one  of  a  series  of  reports  on  the  24  Chief  Financial  Officers  (CFO)  Act  of 
1990  (P.L  101-576, 104  Stat.  2838)  agencies’  fiscal  year  2000  performance  reports  and  fiscal 
year  2002  performance  plans. 

^Observations  on  the  National  Aeronautics  and  Space  Administration’s  Fiscal  Year  1999 
Performance  Report  and  Fiscal  Year  2001  Performance  Plan  (GAO/NSIAD-00-192R,  June 
30, 2000). 

^NASA:  Status  of  Achieving  Key  Outcomes  and  Addressing  Major  Management 
Challenges  (GAO-01-868,  July  31, 2001). 

^NASA’s  fiscal  year  2002  performance  plan  identifies  an  objective  closely  related  to  this  key 
outcome.  That  objective  is  to  operate  the  space  station  to  advance  science,  exploration, 
engineering,  and  commerce.  We  based  our  assessment  on  that  objective. 
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As  agreed,  using  the  selected  key  outcomes  for  NASA  as  a  framework,  we 
(1)  assessed  NASA’s  planned  progress  toward  achieving  these  outcomes, 
and  the  strategies  the  agency  has  in  place  to  achieve  them;  and  (2) 
compared  NASA’s  fiscal  year  2002  performance  plan  with  the  agency’s 
prior  year  performance  plan  for  these  outcomes.  Additionally,  we  agreed 
to  analyze  how  NASA  addressed  its  major  management  challenges, 
including  the  govemmentwide  high-risk  areas  of  strategic  human  capital 
management  and  information  seciuity,  that  we  and  NASA’s  Office  of 
Inspector  General  (OIG)  identified,  i^pendix  I  provides  detailed 
information  on  how  NASA  addressed  these  challenges.  Appendix  11 
contains  NASA’s  comments  on  a  draft  of  our  report 


Results  in  Brief 


NASA  has  improved  its  fiscal  year  2002  performance  plan  and  responded 
to  recommendations  or  suggestions  by  us  and  others  to  make  its  plan 
more  useful — particularly  by  providing  more  comprehensive  explanations 
on  how  it  plans  to  verify  and  validate  performance  data  and  by  better 
portraying  how  its  performance  goals  will  benefit  the  public.  Generally, 
NASA’s  annual  performance  goals  for  its  outcomes  j^rpear  to  be  objective 
and  help  to  measure  progress  toward  the  outcomes.  However,  the  plan 
still  does  not  explain  the  reasons  for  changes  in  performance  goals.  Not 
having  these  explanations  could  hinder  the  ability  to  assess  NASA’s 
performance  over  time. 

Planned  outcome:  Expanding  scientific  knowledge  of  the  Earth  system. 

NASA’s  annual  performance  goals  for  this  outcome  appear  objective;  and 
many  of  the  supporting  performance  indicators  increase  the  measurability 
of  the  performance  goals.  Also,  in  response  to  the  NASA  Advisory 
Council’s  specific  recommendation,  NASA  explains  how  each 
performance  goal  will  benefit  the  public.  However,  these  performance 
goals  and  indicators  do  not  reflect  programs  and  activities  being 
undertaken  with  other  agencies  for  the  strategic  goal,  “observe, 
understand,  and  model  the  Earth  system  to  learn  how  it  is  changing  and 
the  consequences  for  life  on  Earth,”  even  though  NASA’s  recently  updated 
Strategic  Plan  identifies  16  federal  partners  contributing  to  this  goal.® 


*We  based  our  assessment  of  the  Earth  Science  outcome  on  this  strategic  goal.  The 
performance  plan  does  reflect  collaboration  with  other  partners  for  one  strategic  goal 
within  this  outcome  not  covered  by  our  review. 
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NASA’s  strategies  for  achieving  this  outcome  appear  to  be  clear  and 
reasonable. 

•  Planned  outcome:  Expanding  the  commercial  development  of  ^ace. 

NASA  generally  presents  performance  goals  that  appear  objective  and 
measurable  for  this  outcome  and  explains  how  each  wUl  benefit  the 
public.  Although  the  agency’s  strategies  for  meeting  these  performance 
goals  are  generally  clear  and  reasonable,  there  are  some  exceptions.  For 
example,  the  goal  to  develop  and  test  competing  technologies  for  human 
missions  beyond  low  earth  orbit  in  cooperation  with  international  partners 
does  not  address  how  NASA  wiU  actually  test  those  technologies. 

•  Planned  outcome:  Deploying  and  operating  the  International  Space  Station 
safely  and  cost-effectively. 

NASA’s  performance  goals  for  this  outcome  appear  objective  and 
measurable,  and  NASA  explains  how  they  will  benefit  the  public.  However, 
one  of  the  indicators  for  the  safety  performance  goal  is  not  articulated 
understandably,  which  makes  it  difficult  to  ascertain  its  relationship  to  the 
performance  goal  or  assess  its  measurability.  Also,  the  plan  does  not 
clearly  indicate  how  NASA  wiU  ensure  that  the  safety  goal  is  achieved,  nor 
does  it  show  how  the  agency  will  achieve  the  remaining  three  performance 
goals  of  (1)  demonstrating  ^ace  station  progress  and  readiness  at  a  level 
sufficient  to  show  adequate  readiness  in  the  assembly  schedule, 

(2)  successfully  completing  90%  of  the  space  station’s  planned  mission 
objectives,  and  (3)  demonstrating  progress  toward  developing  space 
station  research  hardware.  NASA  does  not  sufficiently  elaborate  on  the 
nature  of  its  space  station  budget  accountability  reforms  mentioned  in  the 
plan,  nor  how  the  reforms  will  address  space  station  cost  growth,  a  long¬ 
standing  and  ongoing  management  problem  Furthermore,  it  does  not 
acknowledge  anywhere  in  the  plan  that  the  International  Space  Station 
cost  control  issue  is  a  management  challenge — although  it  does  so  for 
some  of  the  other  challenges. 

NASA  has  made  improvements  in  its  fiscal  year  2002  performance  plan 
when  compared  to  its  2001  plan.  In  addition  to  its  verification  and 
validation  efforts,  NASA  adds  several  features  that  enhance  the  format 
and/or  content  of  its  performance  plan,  including  a  better  characterization 
of  its  annual  performance  measures  as  “goals”  instead  of  “targets.”  It  also 
adds  discussions  on  how  its  annual  performance  goals  will  benefit  the 
public  and  displays  annual  performance  goals  and  associated  performance 
assessments  for  fiscal  years  1999  to  2002,  to  help  demonstrate  cumulative 
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progress  towards  achievement  of  strategic  goals  and  objectives.  Moreover, 
NASA  adds  an  agencywide  human  capital  objective  and  associated  annual 
performance  goals  and  indicators  to  the  plan.  However,  NASA  can  further 
enhance  the  credibility  of  its  verification  and  validation  efforts  by  being 
more  forthcoming  about  possible  limitations  in  its  performance  data. 
Furthermore,  it  can  relate  its  human  capital  performance  goals  and 
indicators  to  specific  programs  where  issues  of  critical  staffing  shortages 
have  been  identified.  It  can  also  enhance  understanding  of  its  rationale  for 
characterizing  successful  achievement  despite  not  meeting  aU  supporting 
performance  indicators  for  its  goals  by  including  in  the  plan  an 
explanation  for  its  decision  to  use  this  approach.  And  it  can  provide  a 
clear  rationale  for  how  information  technology-related  strategies  and 
programs  will  contribute  specifically  to  the  achievement  of  its  goals  and 
show  the  allocation  of  information  technology-related  dollars  and 
personnel  to  performance  goals. 

In  assessing  how  NASA  addressed  major  management  challenges,  we 
found  the  agency’s  performance  plan  has  strategic  objectives  and 
performance  goals  and  measures  related  to  two  of  our  govemmentwide 
high-risk  areas — strategic  human  capital  management  and  information 
security.  Wlule  NASA’s  inclusion  of  human  coital  as  a  strategic  objective 
is  an  improvement,  it  did  not  tie  its  performance  goals  and  indicators  to 
specific  programs  facing  human  capital  shortages,  such  as  the  space 
shuttle  program.  Regarding  information  security,  the  plan’s  performance 
goals  are  not  fiilly  responsive  to  recommendations  we  made  in  our  1999 
report. 

In  addition,  we  identified  tihree  other  challenges  facing  NASA:  (1) 
correcting  contract  management  weaknesses,  (2)  controlling  International 
Space  Station  costs,  and  (3)  effectively  implementing  the  faster-better- 
cheaper  approach  to  space  exploration  projects.  We  found  that  NASA’s 
performance  plan  has  an  aimual  performance  goal  and  measures  directly 
related  to  the  challenge  of  correcting  contract  management  weaknesses. 
However,  until  NASA’s  Integrated  Financial  Management  System — ^which 
is  central  to  providing  effective  management  and  oversight  over  its 
procurement  dollars — is  operational,  performance  and  cost  assessments 
may  be  incomplete.  The  plan  contains  an  annual  performance  goal  and 
indicator  that  are  indirectly  related  to  the  challenge  of  controlling 
International  Space  Station  costs.  But  the  plan  does  not  indicate  the  extent 
NASA  will  address  space  station  cost  growth.  Furthermore,  the  plan  does 
not  include  goals  and  measures  directly  related  to  effective 
implementation  of  the  faster-better-cheaper  approach  to  space  exploration 
projects. 
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We  provided  copies  of  a  draft  of  this  report  to  NASA  for  its  review  and 
comment.  In  written  comments  on  the  report,  NASA  generally  agreed  with 
the  information  presented  in  the  report  and  noted  several  improvements  it 
would  make  to  its  fiscal  year  2001  performance  report  and/or  fiscal  year 
2003  performance  plan.  NASA’s  written  comments  are  included  as 
appendix  H. 


Background 


GPRA  is  intended  to  shift  the  focus  of  government  decisionmaking, 
management,  and  accountability  fi'om  activities  and  processes  to  the 
results  and  outcomes  achieved  by  federal  programs.  New  and  valuable 
information  on  the  plans,  goals,  and  strategies  of  federal  agencies  has  been 
provided  since  federal  agencies  began  implementing  GPRA.  Under  GPRA, 
annual  performance  plans  are  to  clearly  inform  the  Congress  and  the 
public  of  (1)  the  annual  performance  goals  for  agencies’  major  programs 
and  activities,  (2)  the  measures  that  wUl  be  used  to  gauge  performance, 

(3)  the  strategies  and  resources  required  to  achieve  the  performance  goals, 
and  (4)  the  procedures  that  will  be  used  to  verify  and  validate 
performance  information.  These  annual  plans,  issued  soon  after 
transmittal  of  the  President’s  budget,  provide  a  direct  linkage  between  an 
agency’s  longer-term  goals  and  mission  and  day-to-day  activities.^  Aimual 
performance  reports  are  to  report  subsequently  on  the  degree  to  which 
performance  goals  were  met.  The  issuance  of  the  agencies’  performance 
reports,  due  by  March  31  of  each  year,  represents  a  new  and  potentially 
more  substantive  phase  in  the  implementation  of  GPRA — ^the  opportunity 
to  assess  federal  agencies’  actual  performance  for  the  prior  fiscal  year  and 
to  consider  what  steps  are  needed  to  improve  performance,  and  reduce 
costs  in  the  future.®  NASA’s  final  performance  plan  was  provided  to  the 
Congress  on  July  17, 2001. 

NASA’s  mission  encompasses  human  exploration  and  development  of 
space,  the  advancement  and  commtmication  of  scientific  knowledge,  and 
research  and  development  of  aeronautical  and  space  technologies.  Its 
activities  span  a  broad  range  of  complex  and  technical  endeavors — ^from 
investigating  and  evaluating  the  composition  and  resources  of  Mars;  to 
working  with  international  partners  to  complete  and  operate  the 
International  Space  Station;  to  providing  satellite  and  aircraft  observations 


^The  fiscal  year  2002  performance  plan  is  the  fourth  of  these  annual  plans  under  GPRA. 

®The  fiscal  year  2000  performance  report  is  the  second  of  these  annual  reports  imder 
GPRA. 
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of  earth  for  scientific  and  weather  forecasting  purposes;  to  developing 
new  technologies  designed  to  improve  air  flight  safety. 


Assessment  of  NASA’s 
Performance  Goals 
and  Strategies  for 
Accomplishing 
Selected  Key 
Outcomes 


This  section  discusses  our  analysis  of  NASA’s  performance  goals  and 
measures  and  strategies  the  agency  has  in  place,  particularly  strategic 
human  cjpital  management?  and  information  technology,  for 
accomplishing  the  outcomes.  In  discussing  these  outcomes,  we  have  also 
provided  information  drawn  from  our  prior  work  on  the  extent  to  which 
the  agency  provides  assurance  that  its  reported  performance  information 
will  be  credible. 


Scientific  Knowledge  of  NASA  revised  its  strategic  goal  and  most  of  its  objectives  for  this  key 

the  Earth  System  outcome  in  its  fiscal  year  2002  performance  plan.  We  based  our 

assessment  on  the  strategic  goal  to  observe,  understand,  and  model  the 
Earth  system  to  learn  how  it  is  changing  and  the  consequences  of  change 
for  life  on  this  planet.  The  previous  goal  was  to  expand  scientific 
knowledge  by  characterizing  the  Earth  system. 

NASA’s  performance  plan  does  not  explain  why  it  adjusted  its  previous 
strategic  goal  and  objectives,  nor  does  it  explain  why  it  developed  newly 
formulated  annual  performance  goals  and  supporting  indicators  for  fiscal 
year  2002.  In  discussing  the  reasons  for  these  changes,  NASA  officials  told 
us  that  changes  in  goals  and  measures  were  necessitated  by  the 
formulation  of  new  strategic  science  questions  for  the  Earth  Science 
Enterprise  and  a  refocused  strategic  plan.  In  our  view,  providing  this 
explanation  in  the  plan  would  have  been  useful.  The  performance  plan 
includes  a  chart  that  displays  aimual  performance  goals  and  associated 
performance  assessments  for  fiscal  years  1999  to  2002,  to  help 
demonstrate  cumulative  progress  towards  achievement  of  strategic  goals 
and  objectives  and  to  facilitate  performance  trend  analysis.  However, 
explaining  changes  in  goals  and  measures  over  time  would  improve  the 
performance  trend  analyses,  and  clarify  the  reasons  for  the  new  measures. 
On  the  other  hand,  by  changing  its  performance  goals  annually,  NASA 


’Key  elements  of  modem  strategic  human  capital  management  include  strategic  human 
cj^ital  planning  and  organizational  alignment;  leadership  continuity  and  succession 
planning;  acquiring  and  developing  staffs  whose  size,  skills,  and  deployment  meet  agency 
needs;  and  creating  results-oriented  organizational  cultures. 
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could  hinder  its  ability  to  make  comparisons  between  fiscal  years  and 
effectively  anal5rae  trends  in  performance. 

Generally,  NASA’s  fiscal  year  2002  annual  performance  goals  for  this 
outcome  appear  to  be  objective  and  help  to  measure  progress  toward 
achieving  it.  Specific  ways  to  measure  the  performance  goals  are 
established  through  two  or  more  indicators  that  in  many  cases  provide 
specific,  quantifiable  values  that  increase  the  measurability  of  the 
performance  goal.  An  example  of  such  an  indicator  is:  “Increase  the 
coverage  of  space-based  maps  of  coral  reef  distribution  by  25  percent 
beyond  current  estimates  using  remotely  sensed  imagery.”  NASA  added 
discussions  on  how  each  performance  goal  would  benefit  the  public,  as 
recommended  by  the  NASA  Advisory  Council  in  its  evaluation  of  NASA’s 
fiscal  year  2000  performance  report  In  some  cases  these  discussions 
clearly  articulate  the  benefit  to  the  public;  in  other  cases  they  only  provide 
descriptive  or  background  information.  For  example,  one  performance 
goal  calls  for  increasing  the  understanding  of  stratospheric  ozone  changes, 
as  the  abundance  of  ozone-destroying  chemicals  decreases  and  new 
substitutes  increases.  The  public  benefit  statement,  “Reduction  in 
atmospheric  ozone  amounts  leads  to  an  increased  flux  of  ultraviolet 
radiation  at  the  Earth’s  surface,  with  harmful  effects  on  plant  and  animal 
life  including  human  health,”  explains  the  effect  of  reduced  ozone  amounts 
rather  than  how  the  goal  will  benefit  the  public. 

NASA  indicates  that  it  will  consider  many  of  the  Earth  Science  goals  as 
fuUy  met  if  a  specified  number  of  the  supporting  indicators  (such  as  3  out 
of  4)  are  achieved  in  fiscal  year  2002.  NASA  officials  told  us  that  this 
approach  allows  for  some  flexibility  hi  rating  success.  Specifically,  since 
research  and  development  by  its  very  nature  is  unpredictable,  these 
officials  believe  that,  for  example,  not  meeting  all  indicators  stiU  implies 
significant  progress  in  achieving  scientific  goals.  NASA  could  fully  explain 
in  the  plan  why  it  does  not  believe  it  has  to  meet  all  supporting  indicators. 
This  would  put  its  actual  performance  in  the  proper  perspective.  The 
following  is  one  example  of  such  a  goal: 

“Annual  performance  goal  -  Increase  understanding  of  global 
precipitation,  evaporation  and  how  the  cycling  of  water  through  the 
Earth  system  is  changing  by  meeting  at  least  3  of  4  performance 
indicators. 
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Indicators 


•  Combine  analysis  of  global  water  vapor,  precipitation  and  wind  data 
sets  to  decipher  variations  (and  possible  trends)  in  the  cycling  of 
water  through  the  atmosphere  and  their  relation  to  sea  surface 
temperature  changes. 

•  Analyze  data  from  polar  and  geostationary  satellites  in  a  consistent 
fashion  over  at  least  two  decades  to  evaluate  whether  the 
detectable  moisture  fluxes  are  increasing  beyond  the  expected 
ranges  of  natural  variability. 

•  Determine  the  time  and  spatial  variability  of  the  occurrence  of 
strong  convection  regions,  precipitation  events,  and  areas  of 
drought  to  assess  whether  or  not  there  are  discemable  global 
changes  in  the  distribution  of  moisture  availability  useful  to  food 
and  fiber  production  and  management  of  fresh  water  resources. 

•  Establish  passive  and  active  rainfall  retrievals  of  zonal  means  to 
establish  a  calibration  point  for  long-term  data  records  of  the  World 
Climate  Research  Program,  Global  Precipitation  Climatology 
Project  (GPCP).” 

Concerning  interagency  and  crosscutting  activities,  we  note  that  within 
this  outcome,  NASA  does  not  include  annual  performance  goals  and 
indicators  that  reflect  programs  or  activities  being  undertaken  with  other 
agencies  in  fiscal  year  2002  for  the  strategic  goal  covered  by  our  review, 
even  though  NASA’s  latest  Strategic  Plan  identifies  16  federal  agencies  that 
contribute  to  this  goal.“’  These  agencies  include  the  departments  of 
Defense  and  Commerce,  the  National  Oceanic  and  Atmospheric 
Administration,  and  the  Federal  Emergency  Management  Agency.  If  NASA 
has  planned  collaborative  efforts  related  to  the  performance  goals  and 
indicators  for  this  outcome,  these  are  not  identified  in  the  performance 
plan.  The  performance  plan  provides  an  opportunity  to  evidence 
coordination  among  crosscutting  programs  and  reflect  the  expected 
contribution  of  other  agencies  toward  related  goals. 

NASA  states  that  its  implementation  strategy  for  Earth  Science  research 
programs  is  focused  on  a  set  of  strategic  science  questions  directed  at 
imderstanding  how  the  Earth  system  is  changing  and  the  consequences  for 


“The  perfonnance  plan  does  include  an  objective  and  annual  performance  goals  that 
reflect  collaboration  with  other  agencies  for  one  strategic  goal  within  this  outcome  not 
covered  by  our  review.  That  strategic  goal  is  to  develop  and  adopt  advanced  technologies 
to  enable  mission  success  and  serve  national  priorities. 
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Commercial  Development 
of  Space 


life  on  Earth.  The  plan  indicates  that  these  questions  can  be  addressed 
effectively  with  NASA’s  capabilities,  which  include  observational 
programs,  research  and  analysis,  modeling,  and  advanced  technology.  In 
general,  the  plan  provides  clear  and  reasonable  information  technology. 
One  annual  performance  goal  within  this  outcome  is  to  successfully 
disseminate  Earth  Science  data  to  enable  NASA’s  Earth  Science  research 
and  applications  goals  and  objectives.  To  achieve  this  goal,  NASA  set 
several  specific  performance  indicators,  such  as  increasing  the  number  of 
distinct  NASA  Earth  Observing  System  Data  and  Information  System 
(EOSDIS)  customers  by  20  percent  compared  to  fiscal  year  2001; 
increasing  scientific  and  applications-data  products  delivered  from  the 
Earth  Observing  System  (EOS)  Distributed  Active  Archive  Centers 
(DAAC)  by  10  percent  compared  to  fiscal  year  2001;  and  increasing  the 
number  of  favorable  comments  firom  DAAC  and  other  users  over  fiscal 
year  2001  and  decreasing  the  total  percentage  of  order  errors  by  5  percent 
over  fiscal  year  2001.  These  indicators  provide  specific,  quantifiable  ways 
to  measure  increases  in  output  from  NASA’s  EOSDIS  and  DAACs.  Based 
on  NASA’s  reported  success  in  meeting  similar  indicators  for  fiscal  year 
2000,  these  mdicators  appear  to  be  reasonable  for  fiscal  year  2002. 


In  some  instances,  NASA  revised  or  added  strategic  goals  and  objectives, 
and  annual  performance  goals  within  this  outcome  for  fiscal  year  2002. 

For  example,  the  agency  added  (1)  a  new  strategic  goal  to  provide 
commercial  industry  with  the  opportunity  to  meet  NASA’s  future  launch 
needs,  including  human  access  to  space,  with  new  launch  vehicles  that 
promise  to  dramatically  reduce  cost  and  improve  safety  and  reliability  and 

(2)  a  new  strategic  objective  to  develop  new  capabilities  for  human  space 
flight  and  commercial  applications  through  partnerships  with  the  private 
sector.  Furthermore,  most  of  the  annual  performance  goals  are  either  new 
or  revised  from  targets  in  NASA’s  prior  year  performance  plan.  Further, 
NASA  does  not  provide  any  rationale  or  reasons  for  the  changes  in  the 
plan.  NASA  officials  told  us  that,  generally,  the  changes  were  made  to 

(1)  improve  NASA’s  ability  to  assess  progress  toward  achieving  goals  and 
objectives;  (2)  reflect  commitment  to  safety  and  privatization  efforts;  or 

(3)  reflect  the  broader  scope  of  programs  and  activities  and  shifts  in 
Enterprise  responsibilities.  Again,  providing  such  e3q)lanations  in  the 
performance  plan,  in  our  view,  would  have  been  useful. 

NASA  displays  its  annual  performance  goals  and  associated  performance 
assessments  for  fiscal  years  1999  to  2002  to  help  demonstrate  cumulative 
progress  towards  achievement  of  strategic  goals  and  objectives  and  to 
facilitate  performance  trend  analysis.  As  emphasized  earlier,  changing 
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performance  goals  aimually  could  hinder  NASA’s  ability  to  make 
comparisons  between  years  and  effectively  analyze  trends  in  performance. 
NASA  generally  presents  performance  goals  that  appear  to  be  objective 
and  help  to  measure  progress  toward  this  outcome.  The  agency  also 
explains  how  each  performance  goal  will  benefit  the  public. 

The  strategies  for  achieving  the  performance  goals  are  generally  clear  and 
reasonable.  However,  there  are  some  exceptions.  For  example,  one 
performance  goal  involves  developing  and  testing  -  on  the  ground  and  in 
space — competing  technologies  for  human  missions  beyond  low  earth 
orbit  in  cooperation  with  international  partners.  One  indicator  related  to 
this  performance  goal  involves  organizing  and  conducting  an 
“international  forum”  at  which  preliminary  concepts,  plans,  and 
technology  options  for  future  human/robotic  exploration  and  development 
of  space  would  be  reviewed.  However,  the  indicator  does  not  address  the 
testing  of  competing  technologies.  Also,  NASA  has  a  performance  goal  to 
engage  the  commercial  community  and  encourage  non-NASA  investment 
in  commercial  space  research  by  meeting  at  least  three  of  four 
performance  indicators,  but  the  plan  does  not  state  why  all  of  the 
supporting  indicators  will  not  be  achieved. 


International  Space 
Station’s  Operation,  Cost, 
and  Safety 


Since  the  selected  key  outcome  of  deploying  and  operating  the  ^ace 
station  safely  and  cost  effectively  is  not  included  in  NASA’s  fiscal  year 
2002  performance  plan  as  a  specific  strategic  goal  or  objective,  we  based 
our  assessment  of  it  on  a  related  strategic  objective  in  the  plan — ^to 
operate  the  space  station  to  advance  science,  exploration,  engineering, 
and  commerce. 


NASA  set  four  annual  performance  goals  for  this  outcome  for  fiscal  year 
2002.  The  performance  goals  are  new,  but  the  plan  does  not  provide  any 
rationale  for  the  changes.  'This  is  an  important  omission,  because  as 
pointed  out  earlier,  explaining  changes  in  goals  and  measures  over  several 
years  would  improve  performance  trend  analyses  and  clarify  why  such 
changes  were  made.  In  discussing  the  reasons  for  these  changes,  NASA 
officials  told  us  that  for  the  International  Space  Station,  the  fiscal  year 
2001  goals  and  objectives  relied  on  milestones  that  were  reported  as  either 
complete  or  incomplete,  with  no  provision  for  reporting  progress  toward 
completion.  The  improved  goals  and  objectives  for  fiscal  year  2002  are  tied 
to  milestones  that  allow  reporting  of  progress  in  terms  of  the  percent  of 
the  milestones  completed.  Thus,  the  new  measures  will  provide  greater 
visibility  and  improve  NASA’s  ability  to  assess  progress  toward  achieving 
goals  and  objectives.  In  our  view,  providing  this  explanation  in  the  plan 
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would  have  enhanced  NASA’s  discussion  on  this  outcome.  Also,  similar  to 
the  previous  outcomes,  NASA  displays  performance  goals  and  associated 
assessments  for  fiscal  years  1999  to  2002  to  help  demonstrate  cmnulative 
progress  towards  achievement  of  strategic  goals  and  objectives  and  to 
facilitate  performance  trend  analysis.  However,  changing  performance 
goals  annually  could  hinder  NASA’s  ability  to  make  comparisons  between 
fiscal  years  and  effectively  analyze  trends  in  performance. 

Generally,  NASA’s  four  annual  performance  goals  and  supporting 
indicators  for  the  space  station  outcome  sppear  to  be  objective  and  help 
to  measme  progress  toward  this  outcome,  hi  addition,  the  agency  explains 
how  the  goals  will  benefit  the  public,  stating  how  completing  them 
successfully  will  provide  many  benefits  of  space  research  through  new 
discoveries  and  improved  technological  applications  in  areas  such  as 
medicine,  industrial  processes,  and  fundamental  knowledge.  One 
performance  goal  addresses  space  station  safety.  Specifically,  the  goal  is 
to  demonstrate  space  station  on-orbit  vehicle’s  operational  safety, 
reliability,  and  performance.  The  goal  has  an  indicator  that  provides  for 
zero  safety  incidents  (such  as  no  on-orbit  iryuries),  which  appears 
reasonable.  The  other  indicator  is  not  articulated  understandably,  making 
it  difficult  to  ascertain  its  relationship  to  the  performance  goal  or  to  assess 
its  measurability.  (The  language  is  phrased  as:  “Actual  resources  available 
to  the  payloads  measured  against  the  planned  payload  allocation  for 
power,  crew  time,  and  telemetry. ’0  Also,  the  plan  does  not  clearly  indicate 
the  means  or  strategies  NASA  will  use  to  ensure  that  the  safety 
performance  goal  is  achieved  in  fiscal  year  2002.  Similarly,  the  plan  does 
not  provide  clear  strategies  for  achieving  the  remaining  three  performance 
goals  of  (1)  demonstrating  space  station  progress  and  readiness  at  a  level 
sufficient  to  show  adequate  readiness  in  the  assembly  schedule,  (2) 
successfiiUy  completing  90%  of  the  space  station’s  planned  mission 
objectives,  and  (3)  demonstrating  progress  toward  space  station  research 
hardware  development. 

NASA  does  not  address  i^ace  station  cost  control  as  part  of  this  outcome. 
However,  within  its  commercialization  of  space  outcome,  NASA  set  a 
performance  goal  in  fiscal  year  2002  to  develop  and  execute  a 
management  plan  and  open  future  ISS  hardware  and  service  procurements 
to  cost-effective  innovation  through  competition,  including  launch 
services  and  a  non-govemmental  organization  for  space  station  research. 
NASA’s  indicator  for  the  management  plan  includes  reforms  that 
(1)  strengthen  its  headquarters  involvement,  (2)  increase  communications, 
(3)  provide  more  accurate  assessment,  and  (4)  maintain  budget 
accountability.  NASA  reports  that  the  benefit  to  the  public  of  the 
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management  plan  and  refonns  is  to  ensure  that  future  space  station  costs 
will  remain  within  the  President’s  fiscal  year  2002  budget  plan.  In  our 
view,  NASA’s  discussion  of  the  proposed  management  plan  is  minimal  and 
lacks  specificity.  While  the  management  plan  will  reportedly  include  ISS 
budget  accountability  refonns,  NASA  does  not  elaborate  on  the  nature  of 
such  reforms  or  indicate  to  what  extent  this  plan  will  address  space 
station  cost  growth,  a  long-standing  management  problem.  Furthermore, 
NASA  does  not  acknowledge  anywhere  in  the  performance  plan  that  space 
station  cost  control  is  a  major  management  challenge,  although  it  has  done 
so  for  some  of  the  other  challenges.  In  past  years  and  as  recently  as 
January  2001,  we  have  identified  the  need  to  control  space  station  costs  as 
a  major  management  challenge  for  NASA.”  We  believe  that  the  agency  has 
the  opportunity  to  use  the  completed  management  plan  to  facilitate  the 
development  of  space  station  cost  control  measures  in  future  annual 
performance  plans.  'The  lack  of  performance  measures  that  address  space 
station  cost  control  is  a  shortcoming  that  we  have  identified  in  our 
previous  reviews  of  the  agency’s  annual  performance  plans  and  reports. 


Comparison  of 
NASA’s  Fiscal  Year 
2002  Performance 
Plan  With  the  Prior 
Year  Plan 


For  the  selected  key  outcomes,  this  section  describes  major  improvements 
or  remaining  weaknesses  in  NASA’s  fiscal  year  2002  performance  plan  m 
comparison  with  its  fiscal  year  2001  performance  plan.  It  also  discusses 
the  degree  to  which  the  agency’s  fiscal  year  2002  plan  addresses  concerns 
and  recommendations  by  the  Congress,  GAO,  NASA’s  OIG,  and  others. 


Comparison  of  NASA’s  fiscal  year  2002  performance  plan  differs  in  several  significant 

Performance  Plans  for  ways  from  the  prior  plan.  First,  NASA  portrays  its  planned  efforts  to  verify 

Fiscal  Years  2001  and  2002  validate  performance  information  more  comprehensively  than  in  2001, 

providing  greater  confidence  that  the  performance  results  will  be  credible. 
In  our  review  of  NASA’s  2001  plan,  we  criticized  the  agency  for  not 
explicitly  describing  those  efforts  and  for  not  addressing  data  limitation 
issues  and  problems.  The  2002  plan  includes  specific  agency  data  bases 
and  describes  methods  NASA  will  rely  on  to  support  the  credibility  of 
reported  performance  information.  For  example,  the  plan  references  the 
NASA  Personnel  Payroll  System,  Incident  Reporting  System,  Financial  and 


*  *  Major  Management  Challenges  and  Program  Risks:  National  Aeronautics  and  Space 
Administration  (GAO-01-258,  Jan.  2001). 
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Contractual  Status  of  Programs  System,  and  NASA  Environmental 
Tracking  System  as  specific  data  bases  that  will  be  used  to  verify  and 
validate  performance  data.  The  plan  describes  i^ecific  processes  in  place 
to  support  performance  claims  associated  with  NASA’s  Integrated 
Financial  Management  System,  performance-based  contracts,  contracts 
awarded  to  small  and  small  disadvantaged  businesses,  and  information 
technology.  And  it  describes  a  broad  array  of  methods  to  verify  and 
validate  reported  performance  data  such  as  monthly  reports  from  NASA 
field  centers,  Web  statistics,  count  of  publications,  and  NASA’s  Education 
Computer  Aided  Tracking  System.  Despite  improvements  in  addressing 
data  verification  and  validation  methods,  NASA  still  does  not  acknowledge 
data  limitations  that  could  hinder  performance  measurement.  We  continue 
to  believe  that  NASA  can  further  enhance  the  credibility  of  its  verification 
and  validation  procedures  and  the  usefulness  of  its  performance  data  by 
disclosing  the  expected  limitations  of  its  performance  data  in  its  annual 
performance  plans.  A  March  2001  NASA  Office  of  Inspector  General  report 
identified  limitations  in  NASA’s  fiscal  year  2000  performance  data  and 
indicated  that  NASA  would  discuss  anticipated  data  limitations  in  its 
performance  planning  beginning  with  its  fiscal  year  2002  final  performance 
plan.*^  However,  we  reviewed  the  final  version  of  the  plan,  and  such  a 
discussion  is  not  included. 

Second,  several  added  features  help  to  enhance  the  format  and/or  content 
of  the  fiscal  year  2002  plan.  NASA’s  use  of  “annual  performance  goals”  in 
the  plan  characterizes  its  annual  performance  measures  more  clearly  than 
the  “annual  performance  targets,”  used  in  previous  plans.  The  addition  of 
discussions  on  how  annual  performance  goals  benefit  the  public  helps  to 
better  understand  the  linkage  between  the  goal  and  the  expected  results, 
although  in  some  cases  additional  clarification  could  even  better  convey 
the  actual  benefit  to  the  public.  Value  is  added  to  the  plan  by  NASA’s 
display  of  annual  performance  goals  and  associated  performance 
assessments  for  fiscal  years  1999  to  2002,  to  help  demonstrate  cumulative 
progress  towards  achievement  of  strategic  goals  and  objectives  and 
facilitate  performance  trend  analysis.  However,  changes  in  performance 
goals  over  many  years  could  hinder  NASA’s  ability  to  make  comparisons 
between  years  and  effectively  analyze  trends  in  performance. 


^^Validatkm  And  Verification  Of  Selected  NASA  FY 2000  Performance  Data  Related  To 
The  Government  Performance  And  Results  Act  (GPRA),  (IG-01-020,  Mar.  30,  2001). 
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Also,  this  year’s  plan  includes  an  agencywide  strategic  objective  to  invest 
in  the  use  of  human  coital.  NASA  set  two  annual  performance  goals  for 
fiscal  year  2002  as  progress  towards  this  objective:  (1)  align  management 
of  human  resources  to  best  achieve  agency  strategic  goals  and  (2)  attract 
and  retain  a  workforce  that  is  representative  at  all  levels  of  America’s 
diversity.  However,  there  are  no  human  ci^ital  initiatives  specifically 
linked  to  the  outcomes  or  annual  performance  goals  and  indicators  that 
link  to  specific  programs,  such  as  the  space  shuttle  program.  (See  details 
under  management  challenges.) 

Third,  NASA  could  explain  in  the  plan  what  procedures  it  has  used  to 
characterize  its  performance  goals  as  fuUy  achieved  when  it  has  not  met 
all  of  the  supporting  indicators  for  those  goals.  This  is  particularly  true  for 
the  Earth  Science  outcome.  Providing  such  an  explanation  would  put  the 
actual  performance  in  the  proper  perspective. 

Fourth,  similar  to  the  prior  plan,  the  fiscal  year  2002  plan  stiU  does  not 
provide  a  clear  rationale  for  how  information  technology-related  strategies 
and  programs  will  contribute  specifically  to  achievement  of  NASA’s  goals 
or  show  any  allocation  of  information  technology-related  dollars  and 
personnel  to  performance  goals.  Goals  for  managing  information 
technology  are  generally  stated  in  terms  of  broad  categories  for 
improvement,  such  as  increased  capability  and  efficiency  and  enhanced 
security,  and  include  few  quantitative  measures.  One  exception  is  the  goal 
of  increasing  dissemination  of  Earth  Science  data,  which  is  accomplished 
through  EOSDIS.  The  plan  sets  several  specific  goals  for  increasing  the 
volume  and  distribution  of  Earth  Science  data  and  products. 

Lastly,  in  our  review  of  NASA’s  fiscal  year  2001  plan,  we  suggested  that 
NASA  document  in  its  annual  performance  plans  and  reports,  the  rationale 
for  establishing  new  performance  targets  to  clarify  the  reasons  for  adding 
such  targets.  We  had  noted  that  while  many  of  NASA’s  annual 
performance  targets  were  new  each  year,  there  was  no  stated  basis  for  the 
changes.  In  its  fiscal  year  2002  performance  plan,  NASA  has  formulated 
new  annual  performance  goals  and  has  changed  many  of  its  strategic  goals 
and  objectives  without  including  the  reasons  for  doing  so.  We  continue  to 
believe  that  providing  the  rationale  for  these  changes  wiU  clarify  the 
reasons  for  the  new  goals  and  measures  and  augment  the  value  of 
performance  trend  analyses.  Also,  the  plan  does  not  indicate  whether  or 
not  achieving  any  specific  goals  would  be  negatively  affected  by  external 
factors.  However,  like  the  prior  plan,  the  fiscal  year  2002  plan  states  that 
successful  execution  of  NASA’s  strategic  goals  and  objectives  depends  on 
receipt  of  its  requested  appropriations,  as  well  as  provision  of  funds. 
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materials,  or  services,  that  have  been  committed  to  the  cooperative 
agreements  or  partnerships  which  are  referenced  in  the  performance  plan. 


NASA’s  Efforts  to 
Address  its  Major 
Management 
Challenges  Identified 
by  GAO 


We  have  identified  two  govemmentwide  high-risk  areas:  strategic  human 
capital  management  and  information  secxirity.  Regarding  strategic  human 
capital  management,  NASA’s  fiscal  year  2002  performance  plan  contains  a 
strategic  objective  and  annual  performance  goals  and  indicators  directly 
related  to  human  capital.  Concerning  information  security,  NASA’s 
performance  plan  contains  a  strategic  objective,  an  annual  performance 
goal,  and  indicators  directly  related  to  this  management  challenge.  The 
plan  states  that  safety  and  security  is  one  of  four  areas  on  which  NASA’s 
information  technology  planning  is  focused.  The  fiscal  year  2002  plan  is  an 
improvement  over  the  2001  plan,  which  did  not  include  quantifiable 
measures  for  improving  information  security.  However,  the  plan’s 
performance  goals  do  not  fully  respond  to  the  recommendations  we  made 
in  1999  when  we  reported  that  the  agency  lacked  an  effective  agencywide 
security  program.*^  For  example,  the  plan  sets  a  performance  indicator  of 
completing  90  percent  of  information  technology  security  plans  for  critical 
systems.  However,  we  recommended  that  all  systems  be  formally 
authorized  before  they  became  operational  and  at  least  every 
3  years  thereafter. 


In  addition,  we  have  identified  three  major  management  challenges  facing 
NASA:  (1)  correcting  contract  management  weaknesses,  (2)  controlling 
International  Space  Station  costs,  and  (3)  effectively  implementing  the 
faster-better-cheaper  approach  to  space  exploration  projects.  We  found 
that  NASA’s  performance  plan  contains  an  annual  performance  goal  and 
indicators  directly  related  to  the  problem  of  contract  management  It  is 
important  to  note  that  imtil  NASA’s  Integrated  Financial  Management 
System — ^which  is  central  to  providing  effective  maneigement  and 
oversight  over  its  procurement  dollars — is  operational,  performance 
assessments  relying  on  cost  data  may  be  incomplete  and  full  costing  will 
be  only  partially  implemented. 

While  NASA’s  performance  plan  contains  an  annual  performance  goal  and 
an  indicator  that  indirectly  addresses  the  challenge  of  controlling  space 
station  costs,  it  does  not  indicate  the  extent  that  NASA  wiU  address  space 


^^Informaticm  Security:  Many  NASA  Mission-Critical  Systems  Face  Serious  Risks 
(GAO/AIMD-99-47,  May  20, 1999). 
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station  cost  growth.  As  we  discussed  in  our  January  2001  report,  the 
International  Space  Station  Program  continues  to  face  cost  control 
challenges.  As  with  contract  management,  until  the  Integrated  Financial 
Management  System  is  operational,  NASA  may  lack  the  cost  information 
needed  to  control  space  station  costs. 

Further,  NASA’s  performance  plan  did  not  directly  address  the  challenge 
of  effectively  implementing  the  faster-better-ches^rer  approach  to  space 
exploration  projects.  In  January  2001,  we  also  reported  that  NASA  faces 
significant  challenges  as  it  attempts  to  create  highly  reliable  missions  and 
foster  open  communications  under  the  budget  constraints  of  the  agency’s 
faster-better-cheaper  iq)ace  exploration  strategy.  In  addition,  the  real 
success  of  this  strategy  will  require  a  comprehensive  integration  of  lessons 
learned  fl:om  failures  on  an  agencywide  basis.  Until  NASA  resolves  these 
problems,  its  financial  resources  are  vulnerable  to  inefficient  use. 
Appendix  I  provides  detailed  information  on  how  NASA  addressed  these 
challenges  and  high-risk  areas  as  identified  by  GAO  and  NASA’s  Office  of 
Inspector  General  (OIG). 


Scope  and 
Methodology 


As  agreed,  our  evaluation  was  generally  based  on  the  requirements  of 
GPRA,  guidance  to  agencies  from  the  Office  of  Management  and  Budget 
(0MB)  for  developing  performance  plans  (0MB  Circular  A-11,  Part  2), 
previous  reports  and  evaluations  by  us  and  others,  our  knowledge  of 
NASA’s  operations  and  programs,  our  identification  of  best  practices 
concerning  performance  planning,  and  our  observations  on  NASA’s  other 
GPRA-related  efforts.  We  also  discussed  our  review  with  NASA  officials 
and  with  officials  of  NASA’s  OIG.  The  agency  outcomes  that  were  used  as 
the  basis  for  our  review  were  identified  by  the  Ranking  Minority  Member 
of  the  Senate  Committee  on  Governmental  Affairs  as  important  mission 
areas  for  NASA  and  do  not  reflect  the  outcomes  for  all  of  NASA’s 
programs  or  activities.  The  mayor  management  challenges  confronting 
NASA,  including  the  govemmentwide  high-risk  areas  of  strategic  human 
capital  management  and  information  security,  were  identified  in  our 
January  2001  performance  and  accountability  series  and  high  risk  update, 
and  by  NASA’s  OIG  in  December  2000.  We  conducted  our  review  from 
August  2001  through  October  2001  in  accordance  with  generally  accepted 
government  auditing  standards. 


Agency  Comments 


We  provided  copies  of  a  draft  of  this  report  to  NASA  for  its  review  and 
comment.  In  written  comments  on  the  report,  NASA  generally  agreed  with 
the  information  presented  in  the  report  and  noted  several  improvements  it 
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woidd  make.  Concerning  our  suggestion  that  NASA  could  fully  explain  in 
its  performance  plan  why  it  believes  it  is  not  necessary  to  achieve  all 
performance  indicators  to  demonstrate  annual  performance  goal 
achievement,  NASA  stated  that  it  would  provide  a  statement  containing 
the  supporting  rationale  for  this  approach  in  its  fiscal  year  2003 
performance  plan. 

In  responding  to  our  observation  that  the  fiscal  year  2002  plan  lacked 
sufficient  detail  on  the  nature  of  the  ISS’s  budget  accountability  reforms  or 
how  the  reforms  will  address  longstanding  and  ongoing  management 
problems,  including  cost  growth,  NASA  commented  that  the  reforms  are 
contained  in  its  Program  Management  Action  Plan  that  wiU  be  referred  to 
in  the  fiscal  year  2003  performance  plan.  We  note  that  the  ISS  Program  is 
being  restructured  in  response  to  a  potential  cost  growth  of  $4.8  billion. 
The  restructuring  has  raised  widespread  concerns  about  the  potential 
science  benefits  to  be  realized  by  the  United  States  and  international 
partners.  For  this  reason,  we  believe  it  is  increasingly  important  for 
NASA’s  performance  plan  to  provide  a  clear  path  showing  how  NASA 
intends  to  implement  the  needed  reforms  and  how  the  reforms  wiU  add 
credibility  to  future  ISS  budgets  and  resolve  the  uncertainties  concerning 
the  utility  of  the  ISS. 

NASA  also  commented  on  a  statement  in  our  draft  report  that  the  agency 
does  not  provide  a  clear  rationale  for  how  IT-related  strategies  and 
programs  will  contribute  specifically  to  achievement  of  its  goals  or  show 
the  allocation  of  IT-related  dollars  and  personnel  to  performance  goals. 
According  to  NASA,  the  FT  service  delivery  metric  in  the  plan  aggregates 
each  m^or  IT  service,  such  as  NASA’s  Integrated  Services  Network. 
Remaining  IT  investments  are  embedded  in  each  NASA  project  and 
managed  as  part  of  the  project.  While  this  statement  may  be  true  on  the 
individual  program  level,  it  does  not  address  GPRA  objectives  to 
demonstrate  how  IT-related  strategies  and  programs  contribute 
specifically  to  the  achievement  of  agency  goals  or  show  the  allocation  of 
related  resources. 

Finally,  in  response  to  our  observation  regarding  NASA’s  lack  of 
explanations  in  the  2002  plan  for  annual  performance  changes,  NASA 
agreed  that  including  such  explanations  in  tbe  plan  would  be  useful  and 
that  it  would  characterize  reasons  for  annual  performance  changes  in  its 
2003  performance  plan. 


As  arranged  with  your  office,  unless  you  publicly  announce  its  contents 
earlier,  we  plan  no  further  distribution  of  this  report  until  30  days  after  the 
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date  of  this  letter.  At  that  time,  we  will  send  copies  to  appropriate 
congressional  committees;  the  NASA  Administrator^  and  the  Director, 
Office  of  Management  and  Budget.  Copies  will  also  be  made  available  to 
others  on  request. 

If  you  or  your  staff  have  any  questions,  please  call  me  at  (202)  512-4841. 
Key  contributors  to  this  report  were  Richard  J.  Herley,  Shirley  B.  Johnson, 
Charles  W.  Malphurs,  Christina  Chaplain,  John  de  Ferrari,  Diane  G. 
Handley,  and  Fannie  M.  Bivins. 

Sincerely  yours. 


Allen  Li 

Director,  Acquisition  and 
Sourcing  Management 
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Appendix  I:  Observations  on  NASA’s  Efforts 
to  Address  Its  Major  Management  Challenges 


The  following  table  identifies  the  major  management  challenges 
confronting  NASA,  including  the  govemmentwide  hi^-risk  areas  of 
strategic  human  capital  management  and  information  security.  The  first 
column  of  the  table  lists  the  management  challenges  that  we  and/or 
NASA’s  Office  of  Inspector  General  (OIG)  have  identified.  The  second 
column  discusses  the  extent  to  which  NASA’s  fiscal  year  2002 
performance  plan  includes  performance  goals  and  measures  to  address 
the  challenges  that  we  and  the  OIG  identified.  Of  the  agency’s  fifteen 
major  management  challenges,  its  performance  plan  has  (1)  goals  and 
measures  that  are  directly  related  to  thirteen  of  the  challenges;  (2)  a  goal 
and  measure  indirectly  applicable  to  one  challenge;  and  (2)  no  goals  and 
measures  directly  related  to  one  of  the  challenges. 

Some  of  the  NASA  performance  plan’s  goals  and  measures  we  discuss  may 
not  track  specifically  with  the  key  considerations  of  NASA  OIG’s 
management  challenges  since  the  challenges  themselves  were  presented  in 
a  broad  context  GAO  has  performed  reviews  affecting  a  number  of  the 
areas  mentioned.  This  appendix  highlights  the  results  of  our  assessments, 
where  applicable. 
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Appendix  I:  Observations  on 
NASA’s  Efforts  to  Address  Its 
Major  Management  Challenges 


Table  1 :  Major  Management  Challenges 


Major  management  challenge 

Appiicable  goals  and  measures  in  the  fiscal  year  2002 
performance  plan 

GAO-designated  governmentwide  high  risk 

Strategic  Human  Capital  Management  GAO  has  identified 
shortcomings  at  multiple  agencies  involving  key  elements  of 
modern  human  capital  management,  including  strategic  human 
capital  planning  and  organizational  alignment;  leadership 
continuity  and  succession  planning;  acquiring  and  developing 
staffs  whose  size,  skills,  and  deployment  meet  agency  needs;  and 
creating  results-oriented  organizational  cultures. 

In  August  2000,  we  reported  that  several  internal  NASA  studies 
had  shown  that  the  agency’s  space  shuttle  program’s  workforce 
had  been  affected  negatively  by  NASA’s  downsizing,  much  of 
which  occurred  after  1995.  We  also  reported  that  NASA  had 
begun  taking  actions  to  address  its  shuttle  workforce  problems.Nn 
September  2001 ,  we  reported  in  testimony  that  while  NASA 
continues  to  make  progress  in  revitalizing  the  shuttle  program’s 
workforce,  considerable  challenges  remain.^* 

In  January  2001,  we  also  reported  the  need  to  implement  a 
human  capital  approach  in  NASA’s  workforce  management 
strategies  as  a  major  management  challenge.*’ 


The  plan  contains  a  strategic  objective,  annual  performance  goals 
and  indicators  directly  related  to  this  management  challenge.  The 
plan’s  strategic  objective  is  to  invest  wisely  in  NASA’s  use  of 
human  capital,  developing  and  drawing  upon  the  talents  of  all 
NASA’s  people.  This  objective  applies  to  all  of  NASA’s  workforce 
and  has  two  related  performance  goals:  (1)  align  management  of 
human  resources  to  best  achieve  agency  strategic  goals  and 
objectives  and  (2)  attract  and  retain  a  workforce  that  is 
representative  at  all  levels  of  America’s  diversity.  NASA  further 
indicated  in  discussion  under  the  second  performance  goal:  (1)  in 
fiscal  year  2002,  the  agency  will  develop  a  process  by  which  the 
centers  will  implement  consistent  workforce  planning  resulting  in  a 
plan  for  each  center  that  links  staffing,  funding  resources,  mission 
and  activities,  and  core  competencies;  (2)  NASA  intends  in  fiscal 
year  2002  to  develop  an  initiative  to  enhance  the  centers’ 
recruitment  capabilities,  focusing  on  hiring  new  college  graduates 
to  counterbalance  the  aging  of  the  workforce  due  to  the  halt  in  the 
influx  of  new  college  graduates  during  the  years  of  downsizing; 
and  (3)  NASA  requested  additional  fiscal  year  2002  resources  to 
expand  training  delivery  methods  and  emphasize  the 
development  of  computer-based  training  alternatives  that  can  be 
accessed  at  all  locations  and  levels.  While  NASA’s  inclusion  of 
human  capital  in  its  plan  as  a  strategic  objective  is  an 
improvement,  it  needs  to  relate  Its  human  capital  annual 
performance  goals  and  indicators  to  specific  programs  where 
issues  of  critical  staffing  needs  have  been  identified,  such  as  the 
space  shuttle  program. 
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Appendix  I:  Observations  on 
NASA’s  Efforts  to  Address  Its 
Major  Management  Challenges 


Major  management  challenge _ 

Information  Security:  Our  January  2001  high-risk  update  noted 
that  the  agencies’  and  governmentwide  efforts  to  strengthen 
information  security  have  gained  momentum  and  expanded. 
Nevertheless,  recent  audits  continue  to  show  federal  computer 
systems  are  riddled  with  weaknesses  that  make  them  highly 
vulnerable  to  computer-based  attacks  and  place  a  broad  range  of 
critical  operations  and  assets  at  risk  of  fraud,  misuse,  and 
disruption. 

In  1999,  we  reported  that  NASA  lacked  an  effective  agencywide 
security  program  that  includes  improvements  in  five  areas: 
assessing  risks  and  evaluating  needs,  implementing  policies  and 
controls,  monitoring  compliance  with  policy  and  effectiveness  of 
controls,  providing  computer  security  training,  and  coordinating 
responses  to  security  incidents.*^  The  need  for  such  a  framework 
was  serious;  tests  we  conducted  at  1  of  NASA’s  10  field  centers 
found  that  mission-critical  information  systems  were  vulnerable  to 
unauthorized  access.  We  successfully  penetrated  several  of  these 
systems,  including  one  responsible  for  calculating  detailed 
positioning  data  for  earth-orbiting  spacecraft  and  another  that 
processes  and  distributes  scientific  data  received  from  these 
spacecraft. 


GAO-designated  major  management  challenges _ 

The  Need  to  Correct  Weaknesses  in  NASA’s  Contract 
Management:  \Ne  have  reported  that  NASA’s  contract 
management  is  a  continuing  area  of  high  risk.  Implementation  of 
the  financial  management  system  and  its  Integration  with  full  cost 
accounting  have  been  delayed.  Until  the  Integrated  Financial 
Management  System  Is  operational,  performance  assessments 
relying  on  cost  data  may  be  Incomplete.  We  have  also  reported 
that  NASA  is  continuing  to  rely  on  undefinitized  change  orders— 
that  is,  contract  changes  initiating  new  work  before  NASA  and  the 
contractor  agree  on  a  final  estimated  cost  and  fee— to  complete 
work  on  its  largest  space  station  contract.  We  stated  that  this  is  a 
risky  way  of  doing  business  because  it  increases  the  potential  for 
unforeseen  cost  increases  and  scheduling  delays. 


Applicable  goals  and  measures  in  the  fiscal  year  2002 
performance  plan _ 

The  plan  contains  a  strategic  objective  and  an  annual 
performance  goal  and  Indicators  directly  related  to  this 
management  challenge.  The  plan  states  that  safety  and  security 
is  one  of  four  areas  on  which  NASA’s  information  technology 
planning  Is  focused.  A  performance  goal  Is  established  to 
enhance  information  technology  security  by  meeting  established 
performance  indicators  in  three  critical  areas: 

(1)  vulnerabilities  detected,  (2)  training,  and 
(3)  information  technology  security  plans.  The  plan  includes 
specific  target  percentages  for  reducing  known  system 
vulnerabilities,  training  NASA  personnel,  and  preparing 
information  technology  security  plans.  These  are  all  areas  where 
problems  have  been  documented  at  NASA  in  the  past,  including 
our  review  of  information  security  for  NASA’s  mission  critical 
systems,  completed  in  1999,  The  fiscal  year  2002  plan  is  an 
improvement  over  the  2001  plan,  which  did  not  Include 
quantifiable  measures  for  improving  Information  technology 
security.  However,  the  plan’s  performance  goals  are  not  fully 
responsive  to  the  recommendations  we  made  in  1999.  For 
example,  the  plan  sets  a  performance  indicator  of  completing 
90  percent  of  information  technology  security  plans  for  critical 
systems.  We,  on  the  other  hand,  recommended  that  all  systems 
be  formally  authorized  before  they  became  operational  and  at 
least  every  3  years  thereafter.  Likewise,  NASA’s  performance 
indicator  for  Information  technology  security  training  sets 
employee  training  targets  between  80  and  95  percent  for 
providing  awareness  training  to  different  types  of  NASA 
employees.  We  recommended  that  NASA  establish  a  more 
comprehensive  program  that  would  include  certifying  that  NASA 
civil  servants  and  contract  employees  are  competent  to  discharge 
their  information  technology  security-related  responsibilities. 


The  plan  contains  an  annual  performance  goal  and  indicators 
directly  related  to  this  management  challenge.  The  performance 
goal  Is  to  improve  the  agency’s  financial  management  and 
accountability.  This  goal  has  two  indicators  (1)  cost  at  least 
75  percent  of  the  budget  available  to  cost  during  fiscal  year  2002; 
and  (2)  initiate  the  pilot  phase  of  the  core  financial  project  and  at 
least  one  other  project  module  of  NASA’s  third  attempt  at 
implementing  an  Integrated  Financial  Management  System 
(IFMS).  The  plan  also  states  (1)  an  agency-level  project  team  is  In 
place  at  the  Marshall  Space  Flight  Center,  the  lead  center  for  the 
core  financial  project,  and  the  design  phase  will  begin  in  February 
2002  and  (2)  three  “pathfinder”  projects  have  begun  to  test  out  the 
processes  and  technical  requirements  for  agency-wide 
implementation  of  new  administrative  systems.  The  IFMS  is  key  to 
producing  accurate  and  reliable  information  for  full-cost 
accounting.  Furthermore,  the  plan  does  not  address  the  issue  of 
undefinitized  change  orders. 
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Appendix  I:  Observations  on 
NASA’s  Efforts  to  Address  Its 
Major  Management  Challenges 


Major  management  challenge _ 

The  Need  to  Control  International  Space  Station  (ISS) 
Development  and  Support  Costs:  We  have  reported  that  the  ISS 
program  continues  to  face  cost  control  challenges.®  NASA’s  OIG 
also  reported  that  the  ISS  program  continued  to  experience  cost 
overruns  and  scheduling  delays. 


The  Need  to  Effectively  Implement  the  Faster-Better-Cheaper 
Approach  to  Space  Exploration  Projects,  We  have  reported  that 
NASA  faces  significant  challenges  as  It  attempts  to  create  highly 
reliable  missions  and  foster  open  communications  under  the 
budget  constraints  of  the  agency’s  faster-better-cheaper  space 
exploration  strategy.  In  addition,  real  success  will  require  a 
comprehensive  integration  of  lessons  learned  from  failures  on  an 
agencywide  basis.  Until  NASA  resolves  these  problems,  its 
financial  resources  are  vulnerable  to  inefficient  use. 


Applicable  goals  and  measures  in  the  fiscal  year  2002 
performance  plan _ 

The  plan  contains  an  annual  performance  goal  and  indicator  that 
indirectly  addresses  ISS  cost  control  issues.  The  performance 
goal  Is  to  develop  a  management  plan  and  initiate  ISS  reforms, 
but  the  plan  does  not  indicate  to  what  extent  NASA  will  address 
space  station  cost  growth.  This  challenge  is  discussed  in  detail  in 

the  outcomes  section  of  this  report. _ 

The  plan  does  not  have  goals  and  measures  directly  related  to 
this  management  challenge. 


This  was  designated  as  a  new  major  management  challenge  in 
January  2001. _ 

OIG-designated  major  management  challenges _ 


Safety  and  Mission  Assurance:  NASA’s  OIG  has  reported  that 
safety  and  mission  assurance  has  become  a  serious  challenge  for 
NASA.  Key  considerations  to  ensure  safety  in  future  NASA 
operations  include  (1)  ensuring  an  appropriate  level  of  training  for 
staff  who  conduct  safety  reviews  and  evaluations;  (2)  maintaining 
adequate  safety  reporting  systems;  (3)  ensuring  variances  to 
standard  safety  procedures  are  appropriately  justified,  reviewed, 
and  approved;  (4)  maintaining  an  effective  emergency 
preparedness  program;  (5)  ensuring  NASA  and  contractor 
compliance  with  safety  standards  and  regulations;  (6)  ensuring 
product  safety  and  reliability;  and  (7)  ensuring  the  space  shuttle 
and  the  ISS  maintain  crew  safety. 

In  September  2001,  we  reported  while  NASA  is  making  strides  in 
revitalizing  its  workforce,  its  ability  to  implement  safety  upgrades 
in  a  timely  manner  is  uncertain.^ 


The  plan  contains  strategic  objectives,  annual  performance  goals 
and  indicators  directly  related  to  this  management  challenge.  In 
fact,  NASA’s  plan  contains  a  high  emphasis  on  safety.  The  plan 
has  a  strategic  objective  to  protect  the  safety  of  people  and 
facilities  and  the  health  of  the  workforce.  This  objective’s 
performance  goal  directs  NASA  to  increase  the  safety  of  its 
infrastructure  and  the  health  of  Its  workforce  through  facilities’ 
safety  improvements,  reduced  environmental  hazards,  increased 
physical  security,  enhanced  safety  and  health  awareness,  and 
appropriate  tools  and  procedures  for  health  enhancement.  There 
are  eight  indicators  for  this  performance  goal,  which  include: 

{^)  no  fatalities  will  result  from  NASA  mishaps  and  (2)  per  the 
Federal  Worker  2000  Initiative,  reduce  the  overall  occurrence  of 
Injuries  (due  to  occupational  injury  or  illness)  by  3%  per  year  from 
the  fiscal  year  1997  baseline  to  1 .15  occurrences  per  100 
workers.  The  plan  also  has  strategic  objectives  to  (1)  provide  and 
make  use  of  safe,  affordable,  and  improved  access  to  space  and 
(2)  ensure  the  health,  safety,  and  performance  of  humans  living 
and  working  in  space.  Furthermore,  the  Aerospace  Technology 
Enterprise  has  two  related  strategic  goals  (1)  to  revolutionize 
aviation  -  enable  the  safe,  environmentally-friendly  expansion  of 
aviation  and  (2)  to  advance  space  transportation  -  create  a  safe, 
affordable  highway  through  the  air  and  into  space.  The  plan  also 
states  that  NASA  management.  In  the  interest  of  safety, 
encourages  space  shuttle  program  managers  to  set  aside  metrics 
when  dealing  with  launches  planned  versus  launches  achieved 
during  a  given  fiscal  year.  Our  September  2001  testimony  noted 
that  NASA  is  still  assessing  the  full  package  of  its  planned  space 
shuttle  program  workforce  and  safety  improvements,  and  some 
projects  have  already  encountered  funding  and  scheduling 
problems.  Overcoming  challenges  related  to  the  safety  upgrades 
is  critical  since  NASA  will  be  relying  on  the  space  shuttle  longer 
than  originally  anticipated.  _ 
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Applicable  goals  and  measures  in  the  fiscal  year  2002 
Major  management  challenge _ performance  plan _ 


International  Space  Station:  NASA’s  OIG  has  reported  that  the 
ISS  is  a  significant  management  chailenge  due  to  significant 
problems  related  to  ISS  cost,  contingency  planning,  and  the 
X-38/Crew  Return  Vehicle.  Key  considerations  for  continued  ISS 
assembly  and  operation  are  (1)  managing  the  political,  financial, 
technical,  and  safety  challenges  presented  by  an  international 
partnership;  (2)  overcoming  technical  challenges  inherent  in 
manufacturing,  assembling,  and  testing  complex  hardware  and 
software  components  provided  by  different  nations  and  integrated 
in  space;  (3)  safely  maintaining,  upgrading,  and  operating  a 
structure  as  complicated  as  the  space  station;  and  (4)  maximizing 
the  beneficial  use  of  the  space  station  for  scientific  research  and 
technology  development. 

In  June  2001,  we  reported  that  inadequate  planning  and  design 
led  to  ISS  propulsion  module  failure.® 


NASA  OlG’s  description  of  the  ISS  major  management  challenge 
is  more  broadly  focused  than  our  related  management  challenge. 
(See  discussion  under  GAO-designated  major  management 
challenge,  the  need  to  control  ISS  development  and  support 
costs.)  The  plan  contains  strategic  objectives  and  annual 
performance  goals  directly  related  to  this  management  challenge. 
The  plan  contains  a  strategic  objective  to  conduct  engineering 
research  on  the  International  Space  Station  to  enable  exploration 
beyond  Earth  orbit.  This  objective  has  a  performance  goal  to  test, 
at  the  International  Space  Station,  competing  technologies  for 
human  missions  beyond  low  earth  orbit,  in  cooperation  with  other 
agencies  and  international  partners  and  with  U.S.  industry.  The 
plan  also  has  a  strategic  objective  to  operate  the  International 
Space  Station  to  advance  science,  exploration,  engineering,  and 
commerce.  This  objective  has  four  performance  goals: 

(1)  demonstrate  ISS  on-orbit  vehicle  operational  safety,  reliability, 
and  performance;  (2)  demonstrate  ISS  program  progress  and 
readiness  at  a  level  sufficient  to  show  adequate  readiness  in  the 
assembly  schedule;  (3)  successfully  complete  90  percent  of  the 
ISS  planned  mission  objectives;  and  (4)  demonstrate  progress 
toward  ISS  research  hardware  development.  The  Biological  and 
Physical  Research  Enterprise  has  (1)  strategic  objectives  to 
develop  strategies  to  maximize  scientific  research  output  on  the 
International  Space  Station  and  other  space  research  platforms; 

(2)  a  strategic  objective  to  foster  commercial  research  endeavors 

with  the  International  Space  Station  and  other  assets;  and  (3)  a 
performance  goal  to  highlight  ISS-based  commercial  space 
research  at  business  meetings  and  conferences.  The  plan  also 
has  a  strategic  objective  to  foster  commercial  endeavors  with  the 
International  Space  Station  and  other  assets.  This  objective  has  a 
performance  goal  to  develop  and  execute  a  management  plan 
and  open  future  station  hardware  and  service  procurements  to 
innovation  and  cost-saving  ideas  through  competition,  including 
launch  services  and  a  non-government  organization  for  space 
station  research.  However,  the  plan  does  not  indicate  the  extent 
that  NASA  will  address  space  station  cost  growth.  This  challenge 
is  discussed  in  detail  in  the  outcomes  section  of  this  report. _ 
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Major  management  challenge _ 

Information  Technology:  NASA’s  OIG  has  reported  that 
information  technology  has  become  a  serious  challenge  for 
NASA.  Key  considerations  for  an  effective  information  technology 
program  Include  (1)  ensuring  data  security,  integrity,  and 
application  controls;  (2)  protecting  operations  and 
communications  with  spacecraft;  and  (3)  monitoring  and 
evaluating  the  streamlining  of  operations  through  outsourcing 
information  technology  operations  for  cost  efficiencies, 
dependency  on  the  vendor  for  technological  direction,  vulnerability 
of  strategic  information  to  outsiders,  and  dependency  on  the 
viability  of  the  vendor. 

NASA’s  OIG  reported  that  during  fiscal  year  2000  NASA 
continued  to  have  a  fragmented  information  technology  (IT) 
security  program  without  clear  lines  of  authority,  policies, 
guidelines,  or  enforcement.  The  OIG  reported  that  audits  of 
several  mission  critical  information  systems  disclosed  that  NASA 
had  not  implemented  adequate  basic  controls  in  areas  such  as 
system  access,  protection  of  critical  files,  system  backup  and 
restore  procedures,  privileged  operations  controls,  and  system 
audit  and  monitoring  capabilities. 

(See  discussion  under  governmentwide  high-risk  challenge; 
information  security  for  additional  details.) 


Applicable  goals  and  measures  in  the  fiscal  year  2002 
performance  plan _ 

The  plan  contains  a  strategic  objective,  annual  performance 
goals,  and  indicators  directly  related  to  this  management 
challenge.  The  plan  states  that  safety  and  security  is  one  of  four 
areas  on  which  NASA’s  information  technology  planning  Is 
focused.  The  plan’s  strategic  objective  is  to  enhance  the  security, 
efficiency,  and  support  provided  by  NASA’s  information 
technologies  resources.  The  objective  has  three  performance 
goals  to  (1)  Improve  Information  technology  infrastructure  service 
delivery  by  providing  Increased  capability  and  efficiency  while 
maintaining  a  customer  rating  of  satisfactory;  (2)  enhance  mission 
success  through  seamless,  community-focused  electronic  service 
delivery;  and  (3)  enhance  information  technology  security  by 
meeting  established  performance  indicators  In  three  critical  areas: 
(a)  vulnerabilities  detected,  (b)  training,  and  (c)  information 
technology  security  plans.  The  plan  includes  specific  target 
percentages  for  reducing  known  system  vulnerabilities,  training 
NASA  personnel,  and  preparing  information  technology  security 
plans.  These  are  all  areas  where  problems  have  been 
documented  at  NASA  in  the  past,  including  our  review  of 
information  security  for  NASA’s  mission  critical  systems, 
completed  in  1999.  The  fiscal  year  2002  plan  is  an  Improvement 
over  the  2001  plan,  which  did  not  include  quantifiable  measures 
for  improving  information  technology  security.  However,  the  plan’s 
goals  are  not  as  comprehensive  as  the  recommendations  we 
made  in  1999.  For  example,  the  plan  sets  a  target  of  completing 
90  percent  of  information  technology  security  plans,  including 
authorizations  to  process,  for  Its  critical  systems.  We,  on  the  other 
hand,  recommended  that  all  systems  be  formally  authorized 
before  they  become  operational  and  at  least  every  3  years 
thereafter.  Likewise,  NASA’s  performance  goal  indicator  for 
information  technology  security  training  sets  targets  between 
80  and  95  percent  for  providing  awareness  training  to  different 
types  of  NASA  employees.  We  recommended  that  NASA 
establish  a  more  comprehensive  program  that  would  include 
certifying  that  NASA  civil  servants  and  contract  employees  are 
competent  to  discharge  their  information  technology  securlty- 


related  responsibilities. 
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Major  management  challenge 


Applicable  goals  and  measures  in  the  fiscal  year  2002 
performance  plan 


Procurement:  NASA’s  OIG  has  reported  that  procurement  is  an 
ongoing  challenge  for  NASA.  Key  considerations  for  effective 
procurement  at  NASA  Include  (1)  ensuring  proper  levels  of 
staffing  to  perform  contracting  requirements;  (2)  providing 
sufficient  controls  over  and  monitoring  both  prime  and 
subcontractors;  (3)  Implementing  or  increasing  the  use  of 
innovative  procurement  procedures  such  as  earned  value 
management  and  performance  Incentive  fees;  and  (4)  ensuring 
costs  billed  to  NASA  cost-type  contracts,  due  to  the  changing 
Industry  environment,  are  reasonable  and  allowable. 

In  August  2001,  we  reported  that  NASA’s  inability  to  provide 
timely  data  on  obligations  or  support  for  actual  costs  related  to  the 
space  station  and  shuttle  raises  concerns  about  NASA  being  able 
to  achieve  the  discipline  and  accountability  called  for  by  the 
National  Aeronautics  and  Space  Administration  Authorization  Act 
for  Fiscal  Year  2000  (P.L  106-391)." 


The  plan  contains  a  strategic  objective,  annual  performance 
goals,  and  Indicators  directly  related  to  this  management 
challenge.  The  plan’s  strategic  objective  Is  to  achieve  the  most 
productive  application  of  federal  acquisition  policies.  This 
objective’s  first  performance  goal  Is  to  continue  to  take  advantage 
of  opportunities  for  improved  contract  management  by 
maintaining  a  high  proportion  of  performance  based  contracts 
(PBCs).  This  performance  goal’s  indicator  is  to  maintain  PBC 
obligations  at  greater  than  80%  of  funds  available  for  PBCs.  The 
objective’s  second  performance  goal  is  to  continue  integrating 
small,  small  disadvantaged,  and  women-owned  businesses 
together  with  minority  universities  into  the  competitive  base  from 
which  NASA  can  purchase  goods  and  services.  NASA  further 
Indicated  in  discussion  under  the  strategic  objective  that  (1 ) 
NASA’s  Office  of  Procurement  has  undertaken  proactive 
management  approaches  In  three  key  areas:  human  capital, 
outsourcing  and  oversight,  and  electronic  commerce;  and  (2)  on 
November  2000,  the  Associate  Administrator  for  Procurement  and 
the  Associate  Administrator  for  Safety  and  Mission  Assurance 
jointly  announced  the  establishment  of  the  Surveillance  Planning 
Team  to  provide  policy  direction  and  procedural  guidance  on 
appropriate  surveillance  planning  for  NASA-contracted  work 
based  on  the  risk  associated  with  the  work  and  contractor 


involvement.  The  plan  also  contains  a  performance  goal  to 
continue  implementation  of  planned  and  new  privatization  efforts 
through  the  space  shuttle  prime  contract  and  further  efforts  to 
safely  and  effectively  transfer  civil  service  positions  and 
responsibilities  to  the  space  shuttle  contractor  with  associated 
performance  goal  indicators.  The  plan  also  contains  a 
performance  indicator  to  Initiate  the  pilot  phase  of  the  core 
financial  module  and  at  least  one  other  module  of  the  integrated 
financial  management  system.  If  effectively  Implemented,  this 
system  should  provide  systems  and  processes  to  oversee 
procurement  activities.  However,  this  effort  will  require  continued 
management  attention  to  correct  problems  and  keep  projects  on 
schedule.  Furthermore,  the  plan  does  not  address  the  issue  of 
limiting  NASA’s  frequent  use  of  undefinitized  contract  change 
orders  -  that  is,  unnegotiated  contract  changes.  Relying  on 
unnegotiated  changes  as  a  way  of  doing  business  is  risky 
because  It  Increases  the  potential  for  additional  unanticipated  cost 


irowth. 
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Major  management  challenge _ 

Fiscal  Management  NASA’s  OIG  has  reported  that  fiscal 
management  continues  to  be  a  significant  challenge  for  NASA. 
Key  considerations  to  improved  fiscal  management  include: 

(1)  monitoring  contractor  performance  of  financial  statement 
audits  to  ensure  that  the  statements  are  properly  prepared  and 
thoroughly  reviewed;  (2)  ensuring  adequate  integration  and 
testing  of  newly  developed  automated  accounting  modules  or 
capability;  and  (3)  ensuring  that  NASA  continues  to  properly 
account  for  and  record  financial  transactions  as  new  capability  is 
Implemented. 

In  March  2001,  we  reported  that  NASA’s  fiscal  year  1999 
Statement  of  Budgetary  Resources  was  misstated  by  a  reported 
$644  million  due,  In  part,  to  a  misinterpretation  of  guidance  and 
errors  in  NASA’s  ad  hoc  process  for  generating  budgetary 
Information.*  In  August  2001,  we  reported  on  NASA’s  Inability  to 
provide  timely  data  on  obligations  or  support  for  actual  costs 
related  to  the  space  station  and  shuttle.* 


Applicable  goals  and  measures  In  the  fiscal  year  2002 
performance  plan _ 

The  plan  contains  a  strategic  objective,  annual  performance 
goals,  and  Indicators  directly  related  to  this  management 
challenge.  The  plan’s  strategic  objective  Is  to  manage  NASA’s 
fiscal  and  physical  resources  optimally.  This  objective  has  an 
annual  performance  goal  to  improve  NASA’s  financial 
management  and  accountability.  The  plan  states  two  target 
Indicators  will  measure  the  agency’s  progress  In  meeting  this 
performance  goal:  (1)  cost  at  least  75  percent  of  the  budget 
available  to  cost  during  the  fiscal  year  and  (2)  initiate  the  pilot 
phase  of  the  core  financial  module  and  at  least  one  other  module 
of  the  integrated  financial  management  system  (IFMS).  While  the 
goal  of  Improving  financial  management  and  accountability  is 
commendable,  additional  target  Indicators  should  be  incorporated 
to  measure  NASA’s  progress  In  achieving  its  goal.  Additional 
performance  Indicators  are  needed  to  ensure  that  NASA’s 
systems  and  processes  provide  a  direct  linkage  between  financial 
and  program  operations.  For  example,  both  the  efficiency  and 
effectiveness  measures  should  be  designed  to  assess  the  finance 
organization’s  ability  to  support  NASA’s  mission.  Without 
performance  indicators  that  provide  a  clear  linkage  between 
financial  management  improvement  and  improved  program 
results,  NASA  may  not  be  able  to  accurately  track  its  progress  In 
achieving  its  performance  goals.  Furthermore,  the  implementation 
of  the  IFMS  is  key  to  producing  accurate  and  reliable  Information 
for  full-cost  accounting.  Until  the  system  Is  operational, 
performance  assessments  relying  on  cost  data  may  be 
Incomplete  and  full  costing  will  be  only  partially  implemented. 
Measuring  costs  is  key  to  measuring  performance  In  terms  of 
efficiency  and  cost-effectiveness. 
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Applicable  goals  and  measures  in  the  fiscal  year  2002 
Major  management  challenge _ performance  plan _ 


Program  and  Project  Management  NASA’s  OIG  has  reported  that 
NASA  faces  significant  challenges  in  program  and  project 
management.  Key  considerations  to  effectively  managing  NASA 
programs  include  (1)  improving  planning  to  enable  NASA  to 
accomplish  its  missions  in  the  face  of  budget  and  human  capital 
issues;  (2)  eliminating  duplication  in  programs  and  improving 
coordination  with  other  research  and  development  organizations; 
(3)  ensuring  that  programs  and  projects  accurately  assess  their 
progress  and  successfully  achieve  their  goals;  and  (4)  effectively 
using  technology  developments  to  increase  NASA  productivity. 

Although  we  did  not  identify  this  issue  as  a  management 
challenge,  we  have  reported  in  September  2001  that  there  are 
fundamental  weaknesses  in  the  collection  and  sharing  of  lessons 
learned  in  NASA  by  program  and  project  managers."  In  June 
2001,  we  also  reported  that  the  initial  ISS  propulsion  module 
project  did  not  meet  performance,  cost,  and  schedule  goals 
largely  because  NASA  proceeded  with  the  contractor’s  proposal 
without  following  fundamental  processes  involving  project 
planning  and  execution.' 


The  plan  includes  a  strategic  goal  and  objectives,  annual 
performance  goals,  and  Indicators  directly  related  to  this 
management  challenge.  The  plan  has  a  strategic  goal  to  enable 
NASA’s  strategic  enterprises  and  their  centers  to  deliver  products 
and  services  to  customers  more  effectively  and  efficiently.  This 
goal  has  four  related  objectives:  (1)  enhance  program  safety  and 
mission  success  in  the  delivery  of  products  and  operational 
services;  (2)  improve  NASA’s  engineering  capability  to  remain  as 
a  premier  engineering  research  and  development  organization; 

(3)  capture  engineering  and  technological  best  practices  and 
process  knowledge  to  continuously  improve  NASA’s 
program/project  management;  and  (4)  facilitate  technology 
insertion  and  transfer  and  utilize  commercial  partnerships  in 
research  and  development  to  the  maximum  extent  practicable. 
The  plan  contains  a  performance  goal  to  capture  a  set  of  best 
practices/  lessons  learned  from  each  program,  to  include  at  least 
one  from  each  of  the  four  “provide  aerospace  products  and 
capabilities  crosscutting  process”  subprocesses,  commensurate 
with  current  program  status.  The  plan  contains  a  performance 
goal  to  earn  external  review  rating  of  “achieved  performance 
targef  on  making  progress  in  the  following  area:  design,  develop, 
and  launch  projects  to  support  future  research  in  pursuit  of 
Strategic  Plan  science  objectives.  This  goal  has  an  indicator  to 
meet  no  fewer  than  75  percent  of  the  development  performance 
objectives  for  “major  programs/projects,”  supported  by  completion 
of  performance  objectives  in  majority  of  “other  projects.”  The  plan 
has  a  performance  goal  to  identify  and  evaluate  candidate 
approaches  for  100- to  1000-  day  human  missions  capable  of  a 
5-  to  10-  fold  cost  reduction — ^while  increasing  safety  and 
effectiveness  (compared  to  1990s  projections).  The  plan  contains 
a  strategic  objective  to  meet  sustained  space  operations  needs 
while  reducing  costs.  This  objective  has  two  performance  goals: 
(1)  the  Space  Communications  program  will  conduct  tasks  that 
enable  commercialization  and  will  minimize  investment  in 
government  infrastructure  for  which  commercial  alternatives  are 
being  developed;  and  (2)  performance  metrics  for  each  mission 
will  be  consistent  with  detailed  program  and  project  operations 
requirements  in  project  service  level  agreements. 
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Applicable  goals  and  measures  in  the  fiscal  year  2002 
Major  management  challenge _ performance  plan _ 


Launch  Vehicles:  NASA’s  OIG  reported  on  challenges  in 
(1)  ensuring  the  availability  of  smalt  expendable  launch  vehicles  to 
ensure  schedule  milestones  and  cost-effectiveness  of  NASA 
missions;  (2)  evaluating  whether  NASA  is  providing  the  majority  of 
developmental  funds  and  assigning  technology  rights  to  its 
industry  partners  in  the  development  of  the  new  reusable  launch 
vehicles  in  the  best  interest  of  the  government;  and  (3)  ensuring 
that  plans  are  in  place  and  are  effectively  Implemented  to  address 
shuttle  systems  obsolescence,  logistics  support,  technical/safety 
upgrades,  and  funding. 

Although  we  did  not  identify  this  issue  as  a  management 
challenge,  we  have  reported  and  testified  on  the  factors  that 
contributed  to  the  difficulties  experienced  by  the  X-33  and  X-34 
programs  and  the  steps  needed  to  avoid  repeating  those 
problems  within  the  Second  Generation  Reusable  Launch  Vehicle 
Program.'" 


The  plan  includes  a  strategic  goal  and  objective,  annual 
performance  goal  and  indicators  directly  related  to  this 
management  challenge.  The  Aerospace  Technology  Enterprise 
has  a  strategic  goal  to  provide  commercial  industry  with  the 
opportunity  to  meet  NASA’s  future  launch  needs,  including  human 
access  to  space,  with  new  launch  vehicles  that  promise  to 
dramatically  reduce  cost  and  improve  safety  and  reliability.  The 
strategic  objective  is  to  utilize  NASA’s  Space  T ransportation 
Council  In  combination  with  an  External  Independent  Review 
Team  (EIRT)  to  assure  agency-level  integration  of  near-  and  far- 
term  space  transportation  investments.  The  performance  goal  is 
to  review  results  of  NASA  and  commercial-sector  performed 
launch  system  architecture  studies,  related  requirements,  and 
refinements  in  planned  risk-reduction  investments.  There  are  two 
performance  goal  indicators  (1)  complete  an  assessment  of  the 
space  launch  initiative  architectures  and  requirements  by  EIRT; 
the  EIRT  will  submit  a  written  report  on  their  evaluation  within 
45  days  following  completion  of  the  review;  and  (2)  the  Space 
Transportation  Council  will  review  progress  and  planning  of  the 
space  launch  initiative  at  least  twice  during  the  fiscal  year, 
including  the  report  filed  by  the  EIRT.  However,  NASA’s  plan  does 
not  contain  enough  specific  information  about  how  NASA  will 
ensure  that  the  government’s  best  Interests  will  be  served  in  these 
joint  government-  and  industry-funded  programs. 
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Applicable  goals  and  measures  in  the  fiscal  year  2002 
Major  management  challenge _ performance  plan _ 


Technology  Development  NASA’s  OIG  has  reported  that 
technology  development  has  become  a  serious  challenge  for 
NASA.  Key  considerations  to  effective  technology  development 
Include  (1)  achieving  a  balance  between  scientific  research  and 
technology  development  and  demonstration  projects; 

(2)  continuing  to  refine  the  technology  transfer  process  to  ensure 
that  U.S.  Industry  achieves  the  maximum  benefit  from  the  new 
technologies  identified;  (3)  determining  If  NASA’s  organizational 
structure  effectively  supports  technology  development  and 
transfer;  (4)  forming  Innovative  partnership  arrangements  with 
U.S.  Industry  to  share  both  the  risk  and  costs  of  technology 
demonstration  and  commercialization;  (5)  ensuring  that  NASA 
technology  demonstrations  do  not  unfairly  distort  the  marketplace; 
(6)  ensuring  that  adequate  controls  exist  on  cooperative 
technology  development  programs;  and  (7)  ensuring  adequate 
protection  of  NASA-developed  technology. 


The  plan  has  strategic  goals,  objectives  and  annual  performance 
goals  directly  related  to  this  management  challenge.  The  space 
science  enterprise  contains  a  strategic  goal:  ‘Technology/Long- 
Term  Future  Investments:  develop  new  technologies  to  enable 
innovative  and  less  expensive  research  and  flight  missions.”  The 
strategic  objectives  for  this  goal  are  to:  (1)  acquire  new  technical 
approaches  and  capabilities;  (2)  validate  new  technologies  in 
space;  and  (3)  apply  and  transfer  technology.  The  Earth  Science 
Enterprise  has  a  strategic  goal  to  develop  and  adopt  advanced 
technologies  to  enable  mission  success  and  serve  national 
priorities.  This  goal  has  related  objectives,  performance  goals, 
and  Indicators.  The  plan  has  a  strategic  objective  to  invest  In  the 
development  of  high-leverage  technologies  to  enable  safe, 
effective  and  affordable  human/robotIc  exploration.  This  objective 
has  a  performance  goal  to  begin  development  of  high-leverage 
technologies  to  enable  safe,  effective  and  affordable 
human/robotic  exploration  missions  beyond  low  earth  orbit  (LEO). 
The  plan  also  includes  performance  goals  to:  (1)  test  at  the 
International  Space  Station  competing  technologies  for  human 
missions  beyond  LEO,  In  cooperation  with  other  agencies  and 
International  partners,  and  with  US  industry;  (2)  select  and  fund  at 
least  3-5  proposals  through  the  Human  Exploration  and 
Development  of  Space  Enterprise’s  (HEDS)  Technology  and 
Commercialization  Initiative-focused  research  and  technology 
program  that  feature:  highly  innovative  new  technology 
development  efforts  in  selected  areas  associated  with  human 
safety  and  performance  in  space;  and  (3)  conduct  a  competitive 
solicitation  and  selection  process  that  will  fund  through  the  HEDS 
research  and  technology  program  a  HEDS  Technology  and 
Commercialization  Initiative:  (a)  systems  studies  assessing  the 
commercial  potential  associated  with  various  prospective  HEDS 
infrastructures/capabilities;  and  (b)  new  technology  development 
and  demonstration  efforts  with  potential  longer-term  commercial 
space  value.  The  Biological  and  Physical  Research  Enterprise 
has  a  performance  goal  to  engage  the  commercial  community 
and  encourage  non-NASA  investment  In  commercial  space 
research  by  meeting  at  least  three  of  four  performance  Indicators. 
The  plan  also  has  a  strategic  objective  to  transfer  NASA 
technologies  and  innovations  to  private  Industry  and  the  public 
sector  so  that  the  public  can  benefit  economically  as  well  as 
Intellectually  through  clear,  effective  communications  concerning 
NASA’s  activities. _ 
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Major  management  challenge 


Applicable  goals  and  measures  In  the  fiscal  year  2002 
performance  plan _ 


International  Agreements‘.  NASA’s  OIG  reported  that  international 
agreements  are  needed  to  ensure  effective  and  efficient 
programs.  Key  considerations  include  (1)  program  and  project 
vulnerability  to  schedule  delays  and  cost  overruns  that  require 
diplomatic  rather  than  contractual  solutions;  (2)  security  controls 
on  technology  that  impacts  national  security;  (3)  controls  to 
assure  the  quality  and  timeliness  of  the  goods  and  services 
provided;  (4)  mechanisms  to  assure  a  balance  between  program 
needs  and  national  considerations;  (5)  plans  with  specific  critical 
paths  and  planned  alternative  courses  of  action  to  maintain 
program/project  continuity;  and  (6)  proper  controls  over  access  to 
NASA  facilities  by  foreign  national  visitors. 

Although  we  did  not  identify  this  Issue  as  a  major  management 
challenge,  in  November  1999,  we  recommended  measures  to 
enhance  NASA’s  ability  to  oversee  and  implement  its  export 
controls  of  ISS-related  technologies." 


The  plan  has  a  strategic  objective  and  annual  performance  goals 
directly  related  to  this  management  challenge.  The  plan  has 
performance  goals  to  (1)  collaborate  with  other  federal  and 
International  agencies  In  developing  and  implementing  better 
methods  for  using  remotely  sensed  observations;  (2)  test  at  the 
International  Space  Station  competing  technologies  for  human 
missions  beyond  LEO  In  cooperation  with  other  agencies  and 
international  partners  and  with  US  industry;  (3)  develop  and  test  - 
on  the  ground  and  in  space — competing  technologies  for  human 
missions  beyond  LEO  in  cooperation  with  international  partners; 
(4)  conduct  a  competitive  solicitation  and  selection  process  that 
will  fund  through  the  HEDS  research  and  technology  program  the 
HEDS  Technology  and  Commercialization  Initiative.  The  fourth 
performance  goal  includes  a  performance  indicator  to  develop,  in 
conjunction  with  discussions  with  key  international  space 
organizations,  and  seeks  management  approval  for  an  approach 
for  undertaking  the  formulation  of  international  partnerships  for  the 
development  and/or  demonstration  of  HEDS  capabilities.  The  plan 
also  states  that  in  response  to  recommendations  from  the  NASA 
OIG,  NASA  is  clarifying  the  definition  of  “foreign  national”  In  its 
foreign  visitors  policy  to  ensure  appropriate  and  consistent  use  of 
the  term  in  the  agency’s  foreign  visitors  review  program.  However, 
NASA’s  indicators  for  this  challenge  lack  sufficient  specificity.  For 
example,  the  plan  does  not  specify  actions  taken  to  ensure 
security  controls  on  technology  that  impacts  national  security. 
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Applicable  goals  and  measures  in  the  fiscal  year  2002 

Major  management  challenge _ performance  plan _ 

The  plan  includes  annual  performance  goals  directly  related  to 
this  management  challenge.  The  plan  also  states  that  NASA’s 
strategy  for  achieving  its  environmental  vision  Includes  four  focus 
areas:  prevention,  compliance,  restoration,  and  conservation. 
However,  NASA’s  plan  only  has  performance  metrics  for  the 
areas  of  compliance  and  restoration.  Additional  performance 
indicators  for  the  environmental  focus  areas  of  prevention  and 
conservation  are  needed  so  that  NASA  can  accurately  and  fully 
track  its  progress  toward  achieving  its  environmental  vision.  The 
compliance  related  performance  goal  is  that  NASA  will  increase 
the  safety  of  Its  Infrastructure  and  the  health  of  its  workforce 
through  facilities  safety  improvements,  reduced  environmental 
hazards,  increased  physical  security,  enhanced  safety  and  health 
awareness,  and  appropriate  tools  and  procedures  for  health 
enhancement.  This  performance  goal  includes  an  indicator  to 
reduce  the  level  of  agency  environmental  noncompliance 
incidents  and  releases  in  order  to  achieve  a  5  percent  reduction 
from  the  fiscal  year  2000  level  by  2005.  The  environmental 
restoration-related  performance  goal  is  to  revitalize  agency 
facilities  and  reduce  environmental  liability.  This  performance  goal 
includes  an  indicator  to  reduce  NASA’s  unfunded  environmental 
liability  through  a  long-term  strategy,  annually  investing  an 
amount  of  not  less  than  3-5%  of  the  agency’s  environmental 
liability  in  environmental  compliance  and  restoration  funding.  The 
plan  also  states  management  controls  need  to  be  strengthened  to 
ensure  greater  visibility  of  and  more  consistent  implementation  of 
the  National  Environmental  Policy  Act  (NEPA)  process.  The  plan 
states  review  of  existing  management  controls,  development  and 
advocacy  of  improvements,  and  training  activities  have  been 
planned  and  are  being  initiated.  Furthermore,  the  Aerospace 
Technology  Enterprise  also  has  a  strategic  objective  to  reduce 
emissions  -  protect  local  air  quality  and  our  global  climate  with  a 
related  performance  goal  and  indicators. 

^ Space  Shuttle:  Human  Capital  and  Safety  Upgrade  Challenges  Require 
Continued  Attention  (GAO/NSIAD/GGD-00-186,  Aug.  15, 2000). 

^  Space  Shuttle  Safety:  Update  on  NASA's  Progress  in  Revitalizing  the  Shuttle 
Workforce  and  Making  Safety  Upgrades  (GAO-01-1122T,  Sept  6,  2001). 

Major  Management  Challenges  and  Program  Risks:  National  Aeronautics  and 
Space  Administration  (GAO-0 1-258,  Jan.  2001). 

**  Information  Security:  Many  NASA  Mission-Critical  Systems  Face  Serious 
Risks  (GAO/AIMD-99-47,  May  20, 1999). 

^  (GAO-01-258,  Jan.  2001). 

'  (GAO-01-1122T,  Sept  2001). 

®  Space  Station:  Inadequate  Planning  and  Design  Led  to  Propulsion  Module 
Project  Failure  (GAO-01-633,  Jun.  20,  2001). 

NASA:  International  Space  Station  and  Shuttle  Support  Cost  Limits 
(GAO-Ol-IOOOR,  Aug.  31,  2001). 

‘  Financial  Management:  Misstatement  of  NASA's  Statement  of  Budgetary 
Resources  (GAO-01-438,  Mar.  30,  2001) 

^  (GAO-Ol-IOOOR,  Aug.  31, 2001). 

Survey  of  NASA's  Lessons  Learned  Process  (GAO-01-1015R,  Sept  5,  2001) 


Environmental  Management  NASA’s  OIG  reported  that 
environmental  management  is  a  significant  management 
challenge.  Key  considerations  include:  (1)  prioritizing  and 
addressing  environmental  obligations;  (2)  developing  consistent 
procedures  under  an  agencywide  policy;  and  (3)  negotiating  cost¬ 
sharing  agreements  for  environmental  cleanup  with  previous 
government  and  private  sector  tenants  that  are  also  responsible 
parties. 
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‘(GAO-01-633,  June  2001). 

Space  Transportation:  Status  of  the  X-33  Reusable  Launch  Vehicle  Program 
(GAO/NSIAD-99-176,  Aug.  11, 1999);  Space  Transportation:  Progress  of  the  X-33 
Reusable  Launch  Vehicle  Program  (GAO/T-NSIAD-99-243,  Sept  29, 1999);  and 
Space  Transportation:  Critical  Areas  NASA  Needs  to  Address  in  Managing  Its 
Reusable  Launch  Vehicle  Program  (GAO-01-826T,  June  20,  2001). 

"  Eocport  Controls:  International  Space  Station  Technology  Transfers 
(GAO/NSIAD-00-14,  Nov.  3,  1999). 
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National  Aeronautics  and 
Space  Administration 

Office  of  the  Adminiitrator 

Washington,  DC  20546-0001 


Mr.  Allen  Li 

Director.  Acquisition  and  Sourcing 
Management  Team 

United  States  General  Accounting  Office 
Washington,  DC  20548 


Dear  Mr.  LI: 

NASA  appreciates  the  opportunity  to  comment  on  your  draft  report  entitled  ‘Status 
of  Achieving  Key  Outcomes  and  Addressing  Major  Management  Challenges  (GAO-02- 
1 84).”  We  are  glad  that  our  efforts  to  make  the  Government  Performance  and  Results 
Act  process  more  accountable  and  useful  have  been  successful  as  reflected  in  your 
report.  Although  there  were  no  recommendations,  NASA  would  like  to  provide  additional 
Information  and  clarifications  for  your  final  report. 

In  the  report,  (he  General  Accounting  Office  (GAO)  stated  that  the  Pf  2002 
Performance  Plan  did  not  include  goals  and  measures  directly  related  to  effective 
implementation  of  the  Faster-Better-Cheaper  (FBC)  approach  to  space  exploration 
projects.  NASA  believes  that  FBC  is  a  management  philosophy  that  has  applicability  to 
all  that  the  Agency  does,  if  properly  applied.  The  application  of  FBC  to  programs  and 
projects  must  be  done  on  a  careful,  case-by-case  basis  to  establish  the  risk  posture 
associated  with  a  particular  mission  or  endeavor.  Through  the  Agency’s  program 
management  process  (controlled  by  NASA  Procedures  and  Guidelines  (NPG)  7120.5, 
Program  and  Project  Management)  NASA  continues  to  improve  its  approach  to  applying 
the  principles  of  FBC  with  safety  and  prudent  acceptance  of  mission  risk  as  key  criteria. 
FBC  is  an  overarching  mission  design  philosophy  for  Agencyvride  annual  performance 
goals  and  measures.  We  have  developed  multiyear  trend  metrics  to  measure  Its  effect 
on  Agency  program  approaches.  Such  trend  data  tend  not  to  be  useful  for  measuring 
progress  against  annual  performance  goals. 

GAO  suggested  that  NASA  could  fully  explain  why  it  believes  it  is  not  necessary  to 
achieve  all  indicators  to  demonstrate  significant  performance.  A  statement  provided  to 
GAO  on  October  15. 2001,  details  the  specifics  of  this  rationale.  In  summary,  the 
rationale  is  that  what  counts  in  a  long-term,  basic  research  environment  is  overall 
progress.  Success  or  failure  to  achieve  specific  annual  indicators  is  material  to  the 
evaluation  process  depending  on  the  significance  of  the  specific  indicator  In  the  larger 
context  Related  achievements  may  still  demonstrate  progress  towards  goal 
achievement.  Discretion  has  to  be  applied,  and  not  requiring  100  percent  successful 
achievement  of  all  indicators  when  determining  annual  performance  goal  achievement 
better  reflects  the  nature  of  an  Inherently  unpredictable  research  and  development 
environment  while  encouraging  the  development  of  challenging  metrics.  NASA  will 
insert  a  statement  in  the  FY  2003  Performance  Plan  as  to  the  rationale  supporting  this 
approach. 


NOV  14  2001 
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The  report  suggests  that  NASA  does  not  sufficiently  elaborate  on  the  nature  of  its 
Space  Station  budget  accountability  reforms  mentioned  In  the  performance  plan,  nor 
how  the  reforms  will  address  International  Space  Station  (ISS)  cost  growth,  a  long¬ 
standing  and  ongoing  management  problem.  Elaboration  of  NASA’s  management 
reforms  for  ISS  are  not  contained  in  the  FY  2002  Performance  Plan  but  are  contained  in 
the  Program  Management  Action  Plan  (PMAP)  (Issued  7/01).  NASA’s  development  of 
the  PMAP  Is  recognition  that  grovrth  Is  a  major  management  challenge  for  such  a 
program  as  ISS.  The  PMAP  lays  out  guidelines  for  a  comprehensive  approach  to 
addressing  ISS  cost  growth  and  will  be  expanded  upon  with  details  providing  for 
quantitative  indications  of  successful  Implementation  by  assessing  development  of 
products,  fulfillment  of  actions,  and  Implementation  of  reforms  specified  in  the  plan. 
NASA  will  include  reference  to  the  PMAP  in  the  FY  2001  Performance  Report  and  the 
FY  2003  Performance  Plan. 

GAO  stated  that  NASA  does  not  provide  a  clear  rationale  for  ways  in  which 
Information  Technology  (IT)-related  strategies  and  programs  will  contribute  specifically 
to  the  achievement  of  Its  goals  or  show  the  allocation  of  IT-related  dollars  and  personnel 
to  performance  goals.  The  IT  service  delivery  metric  in  the  Performance  Plan 
aggregates  each  major  service  to  Include  NASA  Integrated  Services  Network  (NISN), 
NASA  ADP  Consolidation  Center  (NACC),  and  Desktop  LAN  &  Voice  Communications 
Services  (ODIN).  Remaining  IT  investments  are  embedded  in  and  are  an  integral  part  of 
each  NASA  project.  Management  of  each  project  (including  embedded  IT)  is  focused  on 
supporting  NASA’s  missions.  The  embedded  IT  is  managed  as  an  integral  part  of  the 
project  Embedded  IT  systems  are  accountable  as  part  of  the  success  of  those  specific 
programmatic  activities,  and  performance  goals  for  the  embedded  IT  are  those 
specifically  set  by  each  program  or  mission  and  may  not  be  an  effective  metric  at  the 
aggregate  level. 

The  GAO  report  calls  on  NASA  to  explain  the  reason  for  changes  from  the 
previous  year's  Performance  Plan.  The  FY  2002  Performance  Plan  was  the  first 
opportunity  to  Incorporate  the  revised  content  of  the  new  NASA  Strategic  Plan  2000. 
However,  we  do  agree  that  explanations  for  annual  performance  changes  would  be 
useful;  therefore,  NASA  will  characterize  reasons  for  change  in  the 
FY  2003  Performance  Plan. 

NASA  will  take  the  more  detailed  comments  in  the  report  under  consideration, 
relative  to  the  details  of  the  specific  program  involved.  Any  resulting  changes  will  be 
reflected  in  the  FY  2003  Performance  Plan. 

NASA  would  like  to  thank  the  General  Accounting  Office  for  the  professional 
manner  in  which  this  review  was  conducted  by  your  staff. 

Sincerely, 

Daniel  R.  Mulville 

Associate  Deputy  Administrator 


(120091) 
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GAO’s  Mission 

The  General  Accounting  Office,  the  investigative  arm  of  Congress,  exists  to 
support  Congress  in  meeting  its  constitutional  responsibilities  and  to  help 
improve  the  performance  and  accountability  of  the  federal  government  for  the 
American  people.  GAO  examines  the  use  of  public  funds;  evaluates  federal 
programs  and  policies;  and  provides  analyses,  recommendations,  and  other 
assistance  to  help  Congress  make  informed  oversight,  policy,  and  funding 
decisions.  GAO’s  commitment  to  good  government  is  reflected  in  its  core  values 
of  accountability,  integrity,  and  reliability. 

Obtaining  Copies  of 
GAO  Reports  and 
Testimony 

■  ' 

The  fastest  and  easiest  way  to  obtain  copies  of  GAO  documents  is  through  the 
Internet.  GAO’s  Web  site  (www.gao.gov)  contains  abstracts  and  full-text  files  of 
current  reports  and  testimony  and  an  expanding  archive  of  older  products.  The 
Web  site  features  a  search  engine  to  help  you  locate  documents  using  key  words 
and  phrases.  You  can  print  these  dociunents  in  their  entirety,  including  charts  and 
other  graphics. 

Each  day,  GAO  issues  a  list  of  newly  released  reports,  testimony,  and 
correspondence.  GAO  posts  this  list,  known  as  “Today’s  Reports,”  on  its  Web  site 
daily.  The  list  contains  links  to  the  full-text  document  files.  To  have  GAO  E-mail 
this  list  to  you  every  afternoon,  go  to  our  home  page  and  complete  the  easy-to-use 
electronic  order  form  foimd  under  “To  Order  GAO  Products.” 

Order  by  Mail  or  Phone 

The  first  copy  of  each  printed  report  is  free.  Additional  copies  are  $2  each.  A 
check  or  money  order  should  be  made  out  to  the  Superintendent  of  Documents. 
GAO  also  accepts  VISA  and  Mastercard.  Orders  for  100  or  more  copies  mailed  to  a 
single  address  are  discoimted  25  percent.  Orders  should  be  sent  to: 

U.S.  General  Accounting  Office 

P.O.  Box  37050 

Washington,  D.C.  20013 

To  order  by  phone:  Voice:  (202)  512-6000 

TDD:  (301)  413-0006 

Fax:  (202)  258-4066 

Visit  GAO’s  Document 
Distribution  Center 

GAO  Building 

Room  1100,  700  4th  Street,  NW  (comer  of  4th  and  G  Streets,  NW) 

Washington,  D.C.  20013 

To  Report  Fraud, 
Waste,  and  Abuse  in 
Federal  Programs 

■ 

Contact: 

Web  site:  www, gao.gov/fraudnet/fraudnet.htm, 

E-mail:  fraudnet@gao.gov,  or 

1-800424-5454  (automated  answering  system). 

Public  Affairs 

Jeff  Nelligan,  Managing  Director,  NelliganJ@gao.gov  (202)  512-4800 

U.S.  General  Accounting  Office,  441  G.  Street  NW,  Room  7149, 

Washington,  D.C.  20548 
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